Emphasizing a portion of the visible content elements of a markup language document

ABSTRACT

Some described embodiments relate to techniques for editing markup elements of a markup language document to emphasize a portion of the visible content elements of the markup language document. The techniques may be implemented in any suitable manner, including via scripting language code (e.g., JavaScript) that is incorporated into the markup language document but is not preconfigured with information regarding the markup language document or any other markup language document. The scripting language code may perform the editing automatically, and based on an automatic analysis of markup elements of the markup language document. Some embodiments may include determining the portion of the markup language document to be emphasized by identifying content of interest to a user, including by determining a set of related content through analyzing a structure of markup elements of the markup language document and/or layout of visible content elements of the markup language document.

RELATED APPLICATIONS

This application claims priority under 35 U.S.C. §119(e) to U.S.Provisional Application Ser. No. 61/930,153, titled “Intelligent ContentLens” and filed on Jan. 22, 2014, the entire contents of which areherein incorporated by reference.

BACKGROUND

Markup languages include markup elements that may serve to identify ordescribe content, including that describe how visible content is to berendered for display. A markup language document may include markupelements describing a content and/or formatting of content of thedocument.

Web pages are an example of markup language documents. A web page may beimplemented as a set of one or more markup language documents, each ofwhich may include content described using Hypertext Markup Language(HTML) elements or Cascading Style Sheet (CSS) elements, and/or elementsof other markup languages. The markup elements of a web page mayidentify text content, image content, scripting language content, orother forms of content and may, for visible content elements, identify amanner in which the visible content elements are to be rendered. Forexample, the markup elements may identify a size or placement of visiblecontent elements such as an image, or may describe a formatting ofvisible content elements such as text. A web browser may, upon downloadof a web page, review the markup elements of one or more markup languagedocuments and render the visible content elements of the web page on adisplay based on the markup elements.

SUMMARY

In one embodiment, there is provided a method comprising, with a viewingapplication processing at least one markup language document, executinginstructions incorporated with the at least one markup language documentto carry out acts. The acts comprise determining a group of one or morevisible content elements, of at least one markup language document, tobe highlighted in a display of the at least one markup languagedocument, and editing markup elements relating to the one or morevisible content elements to visually separate the one or more visiblecontent elements from surrounding visible content elements. The editingcomprises editing at least one first markup element to specify a graphicto display as a background of the one or more visible content elementsand editing at least one second markup element to specify a first bordersurrounding the one or more visible content elements.

In another embodiment, there is provided at least one computer-readablestorage medium having encoded thereon at least one markup languagedocument having incorporated therein executable instructions, whereinthe executable instructions, when executed by at least one processor,cause the at least one processor to carry out a method. The methodcomprises editing markup elements, of at least one markup languagedocument, relating to a set of one or more visible content elements tovisually separate the set of one or more visible content elements fromsurrounding visible content elements. The editing comprises determiningdimensions of a display area including the one or more visible contentelements, calculating, based on the dimensions, first expandeddimensions including the display area and a first margin areasurrounding the display area, calculating, based on the dimensions,second expanded dimensions including the display area, the first marginarea, and a second margin area surrounding the display area, and editingthe at least one markup language document to insert a first new visiblecontent element as a background graphic of the one or more visiblecontent element, the background graphic being a vector graphicspecifying a polygon having dimensions of the display area and having afirst color. The editing further comprises editing the at least onemarkup language document to insert one or more first new markup elementsdefining a second new visible content element positioned overlapping theone or more visible content elements, the second new visible contentelement being a vector graphic including a line having a size and shapespecified according to the first expanded dimensions to surround thedisplay area and having the first color, and editing the at least onemarkup language document to insert one or more second new markupelements defining a third new visible content element positionedoverlapping the one or more visible content elements, the third newvisible content element being a vector graphic including a line having asize and shape specified according to the second expanded dimensions tosurround the display area and the first margin area and having a secondcolor.

In a further embodiment, there is provided an apparatus comprising atleast one processor and at least one computer-readable storage mediumhaving encoded thereon executable instructions that, when executed bythe at least one processor, cause the at least one processor to carryout a method. The method comprises, with a viewing applicationprocessing at least one markup language document, executing instructionsincorporated with the at least one markup language document to carry outacts. The acts comprise determining a group of one or more visiblecontent elements, of at least one markup language document, to behighlighted in a display of the at least one markup language document,and editing markup elements relating to the one or more visible contentelements to visually separate the one or more visible content elementsfrom surrounding visible content elements. The editing comprises editingat least one first markup element to specify a graphic to display as abackground of the one or more visible content elements and editing atleast one second markup element to specify a first border surroundingthe one or more visible content elements.

The foregoing is a non-limiting summary of the invention, which isdefined by the attached claims.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings are not intended to be drawn to scale. In thedrawings, each identical or nearly identical component that isillustrated in various figures is represented by a like numeral. Forpurposes of clarity, not every component may be labeled in everydrawing. In the drawings:

FIGS. 1A-1D are illustrations of examples of markup language documentswith which techniques described herein may operate;

FIG. 2 is a flowchart of an illustrative process for selecting visiblecontent elements to emphasize and for editing a markup language documentto emphasize the visible content elements, which may be implemented insome embodiments;

FIG. 3 is a flowchart of an illustrative process for determining how toselect visible content elements, which may be implemented in someembodiments;

FIG. 4 is a flowchart of an illustrative process for selecting visiblecontent elements, which may be implemented in some embodiments;

FIG. 5 is a flowchart of an illustrative process for evaluating acandidate content element for selection, which may be implemented insome embodiments;

FIG. 6 is a flowchart of an illustrative process for smoothing resultsof a selection process, which may be implemented in some embodiments;

FIG. 7 is a flowchart of an illustrative process for editing a markuplanguage document to emphasize visible content elements in response touser input, which may be implemented in some embodiments;

FIG. 8A is a flowchart of an illustrative process for editing a markuplanguage document to highlight visible content elements, which may beimplemented in some embodiments;

FIGS. 8B-8C are illustrations of highlighting techniques that may beused in connection with the illustrative process of FIG. 8A;

FIG. 9 is a flowchart of an illustrative process for editing a markuplanguage document to magnify visible content elements, which may beimplemented in some embodiments;

FIG. 10 is a flowchart of an illustrative process for incorporating aselection facility and/or emphasis facility into a markup languagedocument, which may be implemented in some embodiments;

FIG. 11 is a diagram of a computer system with which some embodimentsmay operate; and

FIGS. 12-13 are block diagrams of some components of computing deviceswith which some embodiments may operate.

DETAILED DESCRIPTION

Embodiments described herein relate to techniques for editing markupelements of a markup language document to emphasize a portion of thevisible content elements of the markup language document, such as someof the pictures and text of a web page. The techniques may beimplemented in any suitable manner, including via scripting languagecode (e.g., JavaScript) that is incorporated into the markup languagedocument but is not preconfigured with information regarding the markuplanguage document or any other markup language document, thus making thescripting language code reusable between markup language documents,including between markup language documents that are available from orproduced by different publishers (e.g., different web developers orother different authors) or are available from different sources, or areotherwise unrelated documents. The scripting language code may performthe editing automatically, and based on an automatic analysis of markupelements of the markup language document.

In some embodiments, the techniques may include determining the portionof the markup language document to be emphasized by identifying contentof interest to a user, including by determining a set of related contentthrough analyzing a structure of markup elements of the markup languagedocument and/or layout of visible content elements of the markuplanguage document. For example, when a user indicates interest in aspecific visible content element in the markup language document, suchas by hovering a cursor of a pointing device over that specific contentelement, a selection facility may analyze a structure of the markupelements relating to that visible content element and other visiblecontent elements, and/or a layout of that visible content element andother visible content elements, to determine a set of content that isconceptually related. The set of content identified by the selectionfacility may be content that the structure and/or layout of the markuplanguage document suggests a user may want to view together or that apublisher of the markup language document intended to be viewedtogether.

In some embodiments, once the portion of the markup language document isdetermined, the portion may be emphasized in any suitable manner. Insome embodiments, for example, the portion may be emphasized byimproving a visibility of the portion. To improve the visibility of theportion, a format of the portion may be changed to visually separate theportion from surrounding content. By visually separating the portion, itmay be easier for some users to perceive. To visually separate theportion and/or otherwise improve visibility of the portion, a format ofthe portion may be changed to be different from surrounding content. Forexample, font attributes such as color or typeface may be changed, or abackground of the portion may be changed, to be different fromsurrounding content. As another example, a border may be placed aroundthe portion to make it stand out more from surrounding content. As stillanother example of improving the visibility of the portion, the markuplanguage document may be edited to display the portion in a larger size,such as by magnifying the portion. In some embodiments, while thevisibility of the portion is increased, other portions of the markuplanguage document may be maintained in a same format (e.g., not bedisplayed in a larger size) or in a default format. In some embodiments,surrounding content may be grayed out to aid in emphasizing the portion.

Many traditional magnification techniques include magnifying an entirescreen or entire content, such as an entire web page, by the same amountat one time. Though, magnification of portions of content, including webcontent, has also been performed. In a first technique that has beenused for magnifying portions of content, a movable “lens” is fixed tothe cursor of a pointing device, such as a mouse cursor, and a userdrags the lens around a display screen to magnify different content. Ina second technique, a fixed lens is displayed, such as anchored to aside or corner of a screen, that magnifies content displayed in an areaaround a cursor of a pointing device as the user moves the cursor in adisplay screen.

The inventor has recognized and appreciated that thesepreviously-available techniques may have certain disadvantages in someenvironments. For example, for some users who have difficulty viewinginformation on computer screens, such as users with difficulties withtheir eyesight, it may be difficult to distinguish a portion of thecontent from surrounding content, making it difficult, for example, toread a section of a web page. In that case, magnifying an entire screenor an entire content at once may not help those users, as thesurrounding content would still be the same size as the desired content,despite the increase in display size. The first “lens” approach may alsobe difficult for some users to use. When the lens is displayed over thecontent and moves with the cursor of the pointing device, the lens maybe disorienting for a user. While the lens may magnify a section of thecontent and separate that content from its surroundings to make iteasier to view, the lens may also obscure the surrounding content in amanner that may make it difficult for a user to see other content, suchas adjacent content, that the user may also want to view via the lensand thus difficult for the user to move the lens to that other content.The second “lens” approach may also be disorienting for some users. Thelens that is anchored to a side or corner of the screen may not obscurecontent in a way that could make it difficult for a user to operate, butmay still be disorienting for users as they may need to constantly lookback and forth between the lens view on one part of a display screen andanother part of the display screen where the content is displayed andwhere the cursor of the pointing device is moving.

In each of these prior approaches, a graphical magnification techniqueis used by which a picture of the content to be magnified is generatedand the picture enlarged and displayed. The inventor has recognized andappreciated that some of the disadvantages of these prior magnificationtechniques may be mitigated by techniques that instead edit the markuplanguage document to emphasize a portion of the document, such as bydirectly editing markup elements of the document to increase a displaysize of the content or otherwise improve visibility of the content. Theinventor has recognized and appreciated that by editing the markuplanguage document to change a manner in which a portion of the markuplanguage document is displayed, that portion may be emphasized in amanner that may improve visibility of the content without beingdisorienting to a user. By editing the visible content elements withinthe markup language document, the disadvantageous dual-view approachassociated with an anchored lens may be avoided. Further, by editing thevisible content elements in place, the markup elements may be in contextwithout a disadvantageous lens fixed to a cursor and obscuringsurrounding content.

The inventor has also recognized and appreciated that priormagnification techniques are often implemented using special softwarethat is separately executed from software to view a markup languagedocument, and that must be enabled by a user and subsequently disabledby a user to be used. The special software may form a portion of anoperating system or be offered by a third party, for example. Some usersrequire assistance with using computer interfaces, such as because oftheir unfamiliarity with the operations or computers or because ofdisabilities or difficulties of those users. For example, a person whohas poor eyesight and is unfamiliar with the operations of computers mayhave difficulty using the traditional techniques discussed above, suchas an elderly person who may have difficulty viewing content in agraphical user interface and more generally in operating a computer.Triggering such magnification functionality, and subsequently disablingthat functionality when it is no longer necessary, may be a complexoperation that may be difficult for or beyond the capabilities of someusers.

For markup language documents like web pages, a viewing application likea web browser is typically responsible for handling display of contentdescribed by the markup language document. Markup language documentssupport the incorporation of executable instructions, such as scriptinglanguage code, into the markup language document. These instructions maybe executed by the viewing application during processing and display ofthe markup language document. The inventor has recognized andappreciated that such scripting language code could be used to edit amarkup language document to emphasize a portion of the visible contentof the markup language document, including by improving visibility ofthe content. The inventor has additionally recognized and appreciatedthat incorporating functionality for editing the markup languagedocument to improve visibility of content into the markup languagedocument itself may be advantageous for some users, including users withdisabilities or difficulties and/or that have limited experience withcomputers. By incorporating the functionality into the markup languagedocument itself, users would need only to interact with the markuplanguage document within the viewing application and would not need tolearn how to trigger and use other functionality, such as of anoperating system or third party.

Accordingly, discussed below are various embodiments of techniques formoving emphasizing visible content elements of at least one markuplanguage document within a display area in response to user input. Inembodiments, in response to user input (e.g., moving a cursor of apointing device), a selection facility may determine one or more visiblecontent elements that are content of interest to a user. As discussed infurther detail below, while in some embodiments only the content overwhich the cursor is currently positioned may be selected as content ofinterest, in other embodiments the selection facility may analyze astructure and/or layout of the markup language document to determineother content that is related to the content over which the cursor ispositioned. The related content may be, for example, other content thatis conceptually related to the content over which the cursor ispositioned, and/or content that the structure and/or layout suggeststhat a user may want to view, or that a publisher of the markup languagedocument intended to be viewed, together with the content over which thecursor is positioned. By doing so, in these embodiments the content overwhich the cursor is positioned may be emphasized along with the relatedcontent and thereby emphasized in context.

In embodiments, an emphasis facility may then emphasize the contentselected by the selection facility in any suitable manner, including toimprove visibility of the selected content. Examples of emphasistechniques are discussed in more detail below. As one example, theemphasis facility may edit markup elements defining and/or describingthe selected content to improve visibility of the selected content. Theemphasis facility may, for example, change font properties or otherdisplay properties of the selected content. The emphasis facility maychange a background color of, draw a border around, or otherwisehighlight the selected content. The emphasis facility may additionallyor alternatively change a display size of the selected content. Theemphasis facility may, when making these edits to the selected content,not make corresponding changes to other content, such that the selectedcontent may stand out from the original formatting of the other content.The emphasis facility may also, in some embodiments, de-emphasize othercontent, such as by graying out the other content. The emphasis facilitymay make these edits by editing markup elements of the markup languagedocument, including by modifying existing markup elements and/orinserting new markup elements. The emphasis facility may also storeinformation on original markup elements of the selected content and/orother content, and in response to a user input requesting that theemphasis be undone (e.g., when the user is done viewing the emphasizedcontent), the emphasis facility may edit the markup language document toreturn the markup elements to the original.

For ease of description below, many of the examples will be given in thecontext of a single markup language document that is processed by aviewing application and for which visible content elements are displayedin a display area of the viewing application. It should be appreciated,however, that multiple markup language documents may be associated withone another and together describe visible content elements to bedisplayed in a display area of a viewing application, such that theviewing application may process the markup language documents togetherto display the visible content elements. As a specific example, a webpage may be described by multiple markup documents, which may includeone or more HTML documents and one or more CSS documents, among othermarkup documents or other files referenced by the markup languagedocuments. The multiple documents of the web page may refer to oneanother or one (or more) of the documents may refer to the otherdocuments. Accordingly, it should be appreciated that embodiments arenot limited to evaluating one markup language document and any of thetechniques described below (unless indicated otherwise) may be appliedto multiple markup language documents, such as documents that areassociated and together describe a set of visible content elements to bedisplayed in a display area of a viewing application.

The markup language document(s) with which embodiments may operate maybe formatted in any suitable manner and include any suitable content,including any suitable visible content elements and markup elements inany suitable markup language, as embodiments are not limited in thisrespect. In some embodiments, the markup elements may be HypertextMarkup Language (HTML) and/or Cascading Style Sheet (CSS) markupelements and the content may be one or more files of a web page. Suchone or more files of a web page that may be included as markup languagedocuments may include files in any suitable format, including files thatinclude HTML and/or files that include code (e.g., Active Server Pages(ASP), Java Server Pages (JSP), PHP Hypertext Preprocessor (PHP) code,or any other suitable code) that, when executed on a web server and/or aweb browser of a client, output HTML elements or other markup elements.Such one or more files of a web page may additionally or alternativelyinclude one or more style sheets, one or more files linked as definingcontent of a frame or otherwise referenced by an HTML file, or any otherfiles that one of ordinary skill would appreciate could be included in aweb page. Embodiments that operate with web pages are not limited tooperating with any particular type of web page defined in any particularformat, as embodiments may operate with any suitable web page.

Markup language documents like web pages may include visible contentelements to be displayed to a user. Visible content elements may includeany suitable content that may be displayed, including any suitablecontent that may be included in web pages or other markup languagedocuments, as embodiments are not limited in this respect. Examples ofvisible content elements include blocks of text, images, tables, AdobeFlash objects, tables, forms, and other content. Those of skill in theart will appreciate that, in some cases, a markup language document mayinclude non-visible content elements. Non-visible content elements mayinclude elements that organize other content elements, such as containerelements, and may also include types of content elements that mayotherwise be visible but have been configured by markup elements not tobe visible. For example, a developer of a web page may configure apicture or text not to be displayed or configure the picture/text to beincluded in the markup language document at a position that thepicture/text would not appear within a display area of a viewingapplication (e.g., within a viewport of a web browser).

FIGS. 1A and 1B illustrate an example of a markup language document withwhich some embodiments may operate and effects of a selection facilitythat selects related content for emphasis. FIG. 1A illustrates a webpage 100 that includes visible content elements including text andimages. A user manipulates a cursor 102 of a pointing device (e.g.,mouse, trackpad, or other pointing devices) to move within a displayscreen of a computing device on which the web page 100 is displayed. Onthe computing device, a viewing application such as a web browser maycarry out known techniques to process one or more markup languagedocuments describing the web page 100 and render and display visiblecontent elements described by the markup language document(s). The usermay move the cursor 102 within a user interface of the viewingapplication, including within a display area for markup languagedocuments, which may be referred to as the viewport of the web browser.

It should be appreciated that the display area may be displayed on anysuitable type of display screen of any suitable type of computingdevice, as embodiments are not limited in this respect. Though, thetechniques described herein may have particular advantages when used inconnection with computing devices that are personal computers, such asdesktop or laptop personal computers, rather than with mobile computingdevices such as tablets or smartphones.

Movements of the cursor 102 may indicate an interest of the user incontent within the display area. For example, some users move the cursor102 over whatever content the user is currently reading or otherwiseviewing. Accordingly, a position of the cursor 102 may indicate aninterest of the user in a particular content. In the example of FIG. 1A,the user has positioned the cursor 102 over the text 104 “More on NORADtracks Santa app”. Using emphasis techniques described herein, that text104 may be emphasized and visually separated from surrounding content tomake the text 104 easier to view. In the example of FIG. 1A, thatemphasis is performed by adding a border 106 surrounding the text 104.As described in detail below, an emphasis facility may edit the web page100 to insert new markup elements describing the border 106, or modifyexisting markup elements in a way that creates the border 106.

Though, while the text 104 is a specific single content over which thecursor 104 is positioned, it may be the case that isolating text 104 byvisually separating the text 104 from all surrounding content may not bedesirable for a user. For example, text 104 appears in web page 100within a set of content 108 that is conceptually related to text 104.Specifically, all of the content 108 relates to an “app” published byNORAD for tracking Santa's progress on Christmas Eve. The publisher ofthe web page 100 (e.g., the web developer or other author of thecontent, or other party that released the web page 100) clearlypositioned the content 108 together in the web page 100 as a distinctset separate from another distinct set on the left and another distinctset on the right, and may have intended that the content 108 be viewedtogether. Separately, content of the content 108 may be less valuable orharder to understand than if the content 108 were viewed together.Accordingly, while the cursor 104 may indicate that the user is, at atime, specifically interested in the text 104, the user may be viewingthat text 104 in the context of the conceptually-related content 108.Emphasizing the text 104 in isolation would take the text 104 out ofcontext in a way that might be harder for the user to understand or lessdesirable for any other reason. It may be better, then, if the text 104were emphasized in context with other content of the set 108, topreserve the context of the text 104 and other content elements of theset 108.

Accordingly, in some embodiments, using techniques described in detailbelow, a selection facility may analyze the web page 100, including byanalyzing markup elements and/or a layout of the web page 100, todetermine content conceptually related to the text 104. Morespecifically, the selection facility may determine from the position ofthe cursor 102 that text 104 is a first content of interest to the userand may then determine, from an automatic analysis of web page 100, thatthe set 108 is conceptually related to the text 104 or otherwise isarranged in a way that indicates a publisher of the web page 100 mayhave desired for the set 108 to be viewed together. As illustrated inFIG. 1B, the set 108 may then as a group be emphasized, including byvisually separating the set 108 from surrounding content together, as aunit. In the example of FIG. 1B, the set 108 is visually separatedtogether via a border 110 that surrounds the set 108. As described indetail below, an emphasis facility may edit the web page 100 to insertnew markup elements describing the border 110, or modify existing markupelements in a way that creates the border 110.

Through operating a selection facility and an emphasis facility inaccordance with the example of FIG. 1B, a first visible content element(e.g., text) of interest to a user may be emphasized in context withother visible content elements that are related to the first visiblecontent element. It should be appreciated that the border illustrated inFIG. 1B (and in FIG. 1A) is only one example of a type of emphasis thatmay be applied in embodiments. In some embodiments, as discussed above,other types of emphasis may be used, including highlighting, changes infont properties, and/or magnification of content.

FIGS. 1C-1D illustrate an example of a magnification that may be used insome embodiments. FIG. 1C shows a web page 120 that includes visiblecontent elements including text and photos. Web page 120 includes a box122 that includes a set of text and pictures, including a paragraph 124.In the case of FIG. 1C, a user may indicate an interest in paragraph 124and a selection facility may identify that it is content that should beemphasized alone, rather than together with other content of web page120. This may be because, while paragraph 124 is included within box122, box 122 may be too large or includes too many other contentelements to take as a distinct set of content to be emphasized together.It should be appreciated that while it may be helpful at times toemphasize content in context, emphasizing a content element along withtoo many other content items may be the same as not emphasizing thecontent element at all, as the content item may still be difficult for auser to perceive among the other content elements selected with it.Accordingly, in the case of box 122, a selection facility may determinethat the paragraph 124 is to be emphasized in isolation, rather thanalong with all of the other content elements of box 122. Subsequently,the paragraph may be emphasized as illustrated in FIG. 1D.

In the example of FIG. 1D, an emphasis facility has edited the web page120 to emphasize the paragraph 124 that was determined to be of interestto a user and visually separate the paragraph 124 from surroundingcontent. The edited web page 120 includes the text 124 that, in theoriginal web page 120, was included at a smaller size and within theblock 122 along with the other text. In the edited web page 120,however, the text 122 has an increased text size relative to the othertext of block 122, which is shown in the same default size. The web page120 has also been edited to allow the text 124 to expand beyond itsoriginal position, to overlap with other image content that waspreviously positioned to the right of the block 122. The web page 120was also edited to display the text 124 within a new graphic area 126that has a border and a background color that is different from theoriginal background color of block 122. Another graphic element 128 (ormore than one graphic element) has also been added to de-emphasize allcontent other than the text 124, through “graying out” the other contentelements, which emphasizes the text 124 even more. Each of the editsmade to the web page 120 may be made through editing markup elements ofthe web page 120, including by modifying existing markup elements and/orinserting new markup elements. Through these edits, the text 124 appearsto have been excerpted from the web page 120 and displayed floatingabove the remaining content of the web page 120, simulating in a way a“lens” effect that may be achieved through traditional magnificationtechnologies.

Specific techniques for selecting content to be emphasized and forediting a markup language document to emphasize that selected contentare described below. It should be appreciated, however, that each of theexamples below are merely illustrative of embodiments and that otherembodiments are possible.

FIG. 2 illustrates an example of a process by which a markup languagedocument may be edited to emphasize and subsequently de-emphasize aportion of the markup language document, including by changing a mannerof display of some visible content elements of the markup languagedocument. The process 200 may be implemented by one or more facilities,which may be implemented in any suitable manner including as executableinstructions incorporated into the markup language document, such as inscripting language code. In the description of the process 200 below, aselection facility and an emphasis facility are described asimplementing parts of the process 200, but it should be appreciated thatthe division of tasks between facilities is merely illustrative and isnot intended to be limiting. Tasks may be divided between any one, two,or more facilities in any suitable manner in embodiments.

Prior to the start of the process 200, a viewing application (e.g., aweb browser) may respond to user input by downloading a markup languagedocument, such as by responding to user input of an address (e.g., URL)by sending a request to that URL and downloading a markup languagedocument available at that URL, using known techniques. Subsequently,the viewing application may use known techniques to process the markuplanguage document, such as by processing the markup elements describingvisible content elements of the markup language document andrendering/displaying the visible content elements in a display area ofthe viewing application. As part of processing the markup languagedocument, the viewing application may execute executable instructionsincorporated therein. Those skilled in the art will appreciate thatexecutable instructions may be incorporated into a markup languagedocument in a variety of known ways, including by being disposed in themarkup language document or by being referenced by markup elementsdisposed in the markup language document. The process 200 beginsfollowing the processing of the markup language document to display thevisible content elements and execution of the instructions set forththerein, some of which may describe the process 200 and cause theviewing application to carry out the process 200.

The process 200 begins in block 202, in which a selection facilitydetermines a visible content element of interest to a user based oninput from the user. The user input may be any suitable user input, asembodiments are not limited in this respect. For example, the user inputmay be a selection of a visible content element, such as clicking anddragging a cursor of a pointing device. As another example, a user mayclick on the visible content element using a pointing device or tap orotherwise “touch” (including “touch and hold”) the visible contentelement using a pointing device that is a touchpad, touchscreen, orother touch interface. As still another example, a user may use keyboardkeys to toggle a selection between visible content elements, such asusing a “tab” key or spacebar to iterate through visible contentelements of the markup language document to indicate a content elementof interest. As yet another example, the user may move a cursor of apointing device to hover over a visible content element to indicateinterest in that visible content element. In cases in which a particularcontent element is determined from a position of a cursor or a positionof a touch, known techniques may be used to translate a coordinate intoa particular content element. For example, known JavaScript functionssuch as “elementFromPoint( )” may be used. As another example, some webbrowsers may automatically track the content element over which thecursor is currently positioned, and the content element may beidentified at a time by requesting that information from the web browser(or other viewing application).

In examples below, for ease of description the hovering cursor may bedescribed as the manner in which a user indicates interest in a visiblecontent element, but it should be appreciated that embodiments are notso limited and each of these other examples or other ways of indicatinginterest via user input may be used.

In block 204, in response to the user indicating interest in aparticular visible content element, the selection facility may determinewhether that visible content element is included in the markup languagedocument in a set of related visible content elements, such that the setof visible content elements should all be emphasized together toemphasize the content element indicated in block 202 in context. Asdiscussed above, the set of related visible content elements may bevisible content elements that are conceptually related, or otherwiseform a distinct set in the markup language document that is distinctfrom surrounding content elements. When the visible content elementindicated in block 202 appears in such a set of elements that areconceptually related or are otherwise distinct from surrounding content,the user may wish to view the set of content together or a publisher ofthe markup language document may have intended for the set of content tobe viewed together. In such cases, it may be best to emphasize the setof content together and, as such, the selection facility may determinein block 204 whether the content element in block 202 appears in such agroup. The selection facility may make the determination of block 204 inany suitable manner, including according to examples described below inconnection with FIGS. 3-6.

In block 206, for the set of content elements identified in block 204(which may be only the content element indicated in block 202, or thecontent element of block 202 along with related content elements), anemphasis facility stores original markup elements describing the visiblecontent elements of the set. The markup elements that are stored inblock 206 may be the markup elements that define or otherwise describethe visible content elements of the set, such as by describing thecontent elements themselves, the formatting of the content elements fordisplay, the position(s) at which the content elements should bedisplayed, or any other suitable markup elements. The original markupelements are stored in block 206 because, through the process 200, theemphasis facility will edit the markup elements to change a manner ofdisplay of the visible content elements of the set to emphasize thecontent elements. Subsequently, such as after the user has viewed thecontent elements in the emphasized form, the user may wish tode-emphasize the content elements, after which the user may go on toview other content of the markup language document and other visiblecontent elements may be emphasized. As part of de-emphasizing thecontent elements, the emphasis facility may undo the editing that wasdone to the markup language document. The emphasis facility may not haveany prior knowledge, before the beginning of the process 200, about themarkup language document or the particular set of visible contentelements to be emphasized, since the emphasis facility may be unrelatedto the markup language document and may be reusable between markuplanguage documents. To undo the editing performed on the markup languagedocument, then, the emphasis facility may need to additionally edit themarkup language document according to the original markup elements, toremove previously-inserted markup elements and modifypreviously-existing markup elements to return them to the form indicatedby the original markup elements. Alternatively, the emphasis facilitycould remove all markup elements relating to the set of visible contentelements and then insert the original markup elements. In either case,the emphasis facility uses information regarding the original markupelements and thus stores the original markup elements in block 206.

In block 208, the emphasis facility edits the markup language documentto emphasize the set of visible content elements. As should beappreciated, any suitable form of emphasis may be used, as embodimentsare not limited in this respect. In some embodiments, the emphasis maybe done to improve visibility of the visible content elements of theset. Improving the visibility may include visually separating thevisible content elements from surrounding content, to make the visiblecontent elements easier to view as a distinct set for users withdifficulties with their eyesight. Visually separating the visiblecontent elements may include highlighting the visible content elements,such as by adding a new line as a border or by adding a new backgroundimage (e.g., a solid color graphic) that is different from a backgroundof surrounding content. Improving the visibility of the content may alsoinclude changing a color or typeface of text content, such as bychanging a color to have greater contrast with a background color or bychanging a typeface to one that is easier to perceive than an originaltypeface. As still another example, spacing of content may be adjustedor layout may be adjusted in any other way to make the visible contentelements easier to view. As yet another example, the set of visiblecontent elements may be displayed with an increased size relative to adefault display size. Each of these examples may be used alone or in anysuitable combination with each other or with other ways of emphasizingcontent, as embodiments are not limited in this respect. Examples ofways in which an emphasis facility may emphasize content elements aredescribed below in connection with FIGS. 7-10.

In block 208, the emphasis facility performs the emphasis by editing themarkup language document to change a manner of display of the visiblecontent elements of the set. Prior to the editing, the markup languagedocument may not include markup elements describing the emphasized formof the visible content elements or the emphasized form of any othervisible content element of the markup language document. The emphasisfacility edits markup elements relating to the set of visible contentelements by inserting new markup elements describing new formatting tobe applied and/or modifying existing markup elements to change thedescription of formatting provided by those existing markup elements.

As should be appreciated from the foregoing, including the examples ofFIGS. 1A-1D, while the emphasis facility edits the markup languagedocument to emphasize the set of content identified in block 204, theemphasis facility may not edit other content elements or a displaymanner of other content elements. Instead, the emphasis facility maymaintain the default manner of display for the other content elements ofthe markup language document as described by the original, uneditedmarkup language document. In some embodiments, the emphasis facility mayinsert a new visible content element that is disposed on top of othervisible content elements, such as a new partially-transparent graphicthat effectively “grays out” the surrounding content. The newpartially-transparent graphic may be specified in any suitable manner,including via an image added to the markup language document or bysetting a formatting of an element (e.g., a “div” element), that isnewly-inserted or was previously included in the document, to include apartially-transparent color as a background or as another attribute.This new visible content element may be separate from the other,original visible content elements of the markup language document thatsurround the set of visible content elements being modified.Accordingly, in embodiments in which the partially-transparent graphicis added, the addition of the new partially-transparent graphic may notinvolve editing of the markup elements related to the other, originalvisible content elements surrounding the set being emphasized. As such,in some embodiments markup elements relating to the other, originalvisible content elements may not be modified during emphasis of visiblecontent elements of the set.

Though, as discussed below in greater detail, in some embodiments achange made to markup elements for the set of visible content elementsto emphasize one or more the set of visible content elements couldaffect a manner of display of other, surrounding content elements. Forexample, a change to a position of one visible content element of theset as part of the emphasizing may trigger a change to a position of asecond visible content element outside the set that, unlesscounteracted, may undesirably change the position of the second visiblecontent element. Other changes to markup elements for the set to beemphasized may also undesirably impact surrounding visible contentelements. Accordingly, in some cases, changes may be made to markupelements for other visible content elements that are not to beemphasized to preserve a default manner of display of the other visiblecontent elements.

Following the editing of block 208, the viewing application in which themarkup language document is displayed updates the display of the markuplanguage document and, in accordance with the edited markup elements,displays the set of visible content elements in the emphasized format.The updating of the display by the viewing application may be performedautomatically by the viewing application in some cases, such asaccording to a regular update interval, while in some embodiments theemphasis facility may trigger the updating by communicating a requestfor updating to the viewing application.

At some time following the editing of block 208, the user may requestthat the emphasis be removed. For example, the user may view the set ofvisible content elements in the emphasized format and, having viewedthat content, desire to view other content of the markup languagedocument, such as by viewing that other content in an emphasized form.As such, the user may provide input requesting that the emphasis beremoved. The user input that is received may be any suitable user input,including any of the examples of user input described in connection withblock 202. In some cases, for example, the user input may be a userinput that indicates another visible content element, such as moving acursor of a pointing device away from the visible content elementindicated in block 202 and over another element. As another example, theuser input may be a keyboard entry, such as an “escape” key or spacebarinput that the emphasis facility detects as a request to end theemphasis, or another keyboard entry (e.g., a directional arrow input, apage up/down input, or another input) that indicates a selection of anext content element. In block 210, the emphasis facility detectswhether such input has been received. If not, the emphasis facilityloops back and waits for the user input. If the user input has beendetected, however, then in block 212 the emphasis facility edits themarkup language document again to change the markup elements for the setof visible content elements that were emphasized. The edits made by theemphasis facility in block 212 undo the edits made in block 208. Asdiscussed above in connection with block 206, in block 212 the emphasisfacility uses information regarding the original markup elements storedin block 206 to restore markup elements describing the set of visiblecontent elements to their original form. In cases in which markupelements relating to other, surrounding visible content elements weremade to preserve the original formatting of those other elements, asdescribed above, information stored in block 206 regarding these othervisible content elements may also be used by the emphasis facility torestore the markup elements for those other visible content elements totheir original form. The visible markup elements, following the editingof block 212, will be restored to the default formatting for each of thevisible content elements. In some embodiments, the editing of block 212may result in markup elements that are identical to the markup elementsprior to the beginning of the process 200.

Following the editing of block 212, the process 200 ends. As a result ofthe process 200, a manner of display for one or more visible contentelements has been changed from the default to another manner of displaywhich may be easier for some users to view, and subsequently returned tothe default manner of display. Following the process 200, a user mayindicate a new visible content element and the process 200 may berepeated for a new set of one or more visible content elements, toemphasize (and subsequently de-emphasize) those other visible contentelements and make those other visible content elements temporarilyeasier for some users to view.

As should be appreciated from the foregoing discussion, there are anumber of ways in which a selection facility may determine whether aparticular visible content element is a member of a set of relatedvisible content elements (e.g., as in block 204 of FIG. 2) and a numberof ways in which an emphasis facility may edit a markup languagedocument to emphasize a set of one or more visible content elements(e.g., as in block 208 of FIG. 2). Illustrative examples of ways inwhich these functions may be performed are described below.

As discussed above, a selection facility may analyze a markup languagedocument to determine whether a particular visible content element, suchas one indicated by user input, is a member of a set of related visiblecontent elements. The selection facility that performs the analysis maybe independent of any particular markup language document and maytherefore perform an analysis that is independent of any particularmarkup language document, but instead may consider the same factors orother information across multiple different markup language documentsavailable from multiple different publishers of markup languagedocuments. Though, in some embodiments, a publisher of a markup languagedocument (e.g., a web developer or other author of a markup languagedocument) may wish to configure his or her markup language document toinfluence the analysis performed by the selection facility. Accordingly,in some embodiments, as part of determining whether a visible contentelement is a member of a set of related visible content elements, theselection facility may examine the markup language document forconfiguration information that may indicate whether the visible contentelement is a member of a set.

FIG. 3 illustrates an example of a process by which a selection facilitymay analyze a markup language document to determine whether it includesconfiguration information indicating how a selection process should becarried out for a particular visible content element. Prior to the startof the process 300, a markup language document is processed by a viewingapplication and displayed, and user input is received by a selectionfacility indicating one visible content element in which a user may beinterested.

The process 300 begins in block 302, in which the selection facilitydetermines whether the markup language document includes informationdefining how related content should be selected. The information may bein any suitable format, as embodiments are not limited in this respect.In some embodiments, for example, configuration information may beincluded in a configuration file referenced by the markup languagedocument, which may indicate a particular set of content elements orattributes of content elements that are to be selected when a selectionprocess is carried out. For example, a web developer that created amarkup language document may have segmented the content of the markuplanguage document into distinct sets using a table arrangement in whicheach cell of the table is identified using an HTML “td” element. Eachtable cell may include one or more visible content elements. Theconfiguration file may specify that, when any visible content element isbeing analyzed, the corresponding “td” element for the table cell inwhich that visible content element is positioned should be selected andused to specify the set of related content. That is, the corresponding“td” element should be identified, and all of the visible content withinthat “td” cell should be identified as the set of related content.Through this information set out in the configuration file, theselection file may carry out a relatively simple analysis of the markuplanguage document as compared to examples below: the corresponding “td”element is identified and all visible content included therein areselected for inclusion in the set (or the “td” element is selected asthe element defining which visible content elements are included in theset, which are all the visible content elements included within that“td” container). As another example, the configuration file may identifyattributes that have been appended to markup elements defining theelements that are to be selected. The configuration file may specify,for example, that a customized attribute (e.g., a custom “selectThisOne”attribute) has been set as “True” or otherwise set on some markupelements. When the selection facility is determining whether there isrelated content for a first visible content element, in this example,the selection facility examines the markup language document for whetherthere is an element describing a container that includes the firstvisible content element and has the customized attribute. If so, thatelement should be selected and used to identify the set of relatedcontent, which would be all the visible content elements included withinthat container. As a similar example of a configuration file describingattributes, pre-existing, standardized attributes rather than customizedones may be used. For example, the Web Accessibility Initiative'sAccessible Rich Internet Applications (WAI-ARIA) project describes astandard set of metadata that may be added to markup language documentsto define how the markup language document is arranged, which may beuseful to accessibility programs that are attempting to automaticallyanalyze the markup language document. The configuration file may explainthat WAI-ARIA content is used, or explain which WAI-ARIA metadata shouldbe used by the selection facility to identify a set of related content.For example, the configuration file may look for WAI-ARIA attributesthat identify a WAI-ARIA “region” and select the visible contentelements within that “region” as the set of related content.

In other embodiments, rather than being included in a configuration fileor otherwise separated within the markup language document, theselection facility may determine whether configuration information isdisposed within the markup elements of the markup language document. Forexample, the selection facility may be configured to use WAI-ARIAmetadata where available and thus may examine the markup languagedocument for whether it includes WAI-ARIA metadata.

If the selection facility determines in block 302 that the markuplanguage document includes configuration information defining howrelated content is to be selected, then in block 304 the selectionfacility selects related content based on the configuration informationin the markup language document. As described briefly above, this mayinclude identifying based on the configuration information a markupelement in the markup language document that defines a group of contentthat includes a first visible content element (e.g., one indicated byuser input) and that may include other visible content elements relatedto that first visible content element.

If, however, the selection facility determines in block 302 that themarkup language document does not include configuration informationrelating to the selection process, then in block 306 the selectionfacility proceeds by performing an automated analysis of the markuplanguage document. The selection facility performs the automatedanalysis to determine whether the markup language document includesother visible content elements that are related to a first visiblecontent element (e.g., one indicated by user input). The automatedanalysis may include evaluating a structure of markup elements of themarkup language document and/or a layout of visible content elements asdefined by the markup elements. As a result of the automated analysis,the selection facility may determine whether the first visible contentelement is to be selected alone or if it is to be selected along withone or more other visible content elements.

Once the set of visible content elements to be selected is determined ineither block 304 or block 306, the process 300 ends. As a result of theprocess 300, a set of related content has been selected, which mayinclude a single visible content element that was indicated by userinput or may include that single visible content element along withother visible content elements that are related to the single visiblecontent element. Following the process 300, an emphasis facility mayedit the markup language document to change a manner of display of thosevisible content elements to emphasize the visible content elements ascompared to surrounding visible content elements. The selection facilitymay aid this process with the selection of related content (if any) forthe single visible content element indicated by the user input to enablethe emphasis to be applied to that single visible content element incontext with other visible content elements.

In embodiments in which the selection facility carries out an automatedanalysis of the markup language document to determine whether the markuplanguage document includes other visible content elements related to afirst visible content element (e.g., one indicated by user input), theautomated analysis may be carried out in any suitable manner. FIGS. 4-6illustrate examples of ways in which the automated analysis may beperformed.

It should be appreciated that the automated analysis is carried outbecause the selection facility is not preconfigured with informationregarding the markup language document being identified that relates tohow content is laid out or described in the document, or about whatcontent is related to which other content. Prior to the automatedanalysis, the selection facility may not include any informationregarding relationships between content elements of the markup languagedocument to be analyzed. The selection facility may instead beconfigured with information regarding how publishers of markup languagedocuments typically arrange related content and/or how users typicallyview content of markup language documents or identify related content.The information may relate to a layout of content in the markup languagedocument or a structure of markup elements in the markup languagedocument.

In some embodiments, the selection facility may carry out the automatedanalysis to identify a best “container” of content elements to select.Those skilled in the art will appreciate that markup elements of amarkup language document are arranged hierarchically, with “parent”markup elements at one level of the hierarchy containing “child” markupelements that are arranged lower in the hierarchy and within the“parent” markup elements. The content elements of the markup languagedocument are arranged similarly, with content elements (visible or not)described by markup elements at one level of the hierarchy containingcontent elements (visible or not) described by markup elements lower inthe hierarchy. The hierarchy of content elements can thus be considereda hierarchy of containers. When a visible content element is identifiedby user input, the selection facility may identify the position in thehierarchy of that visible content element and then identify each of thecontainers for that visible content element in the hierarchy. Each ofthe content elements identified in this way may then be evaluated in theautomated analysis to determine whether they best represent a set ofrelated visible content elements. One content element is selected fromthat analysis, which may be either the visible content element indicatedby the user input or one of the containers located above the visiblecontent element in the hierarchy. If one of the containers is selected,then the visible content elements of that container are identified asthe set of related content elements.

FIG. 4 illustrates an example of such a process for automated analysisof content elements defined by a hierarchical arrangement of markupelements. Prior to the process 200 of FIG. 4, a viewing application maydownload and process a markup language document into which isincorporated a selection facility, and execute the selection facility.The selection facility may receive an indication of a first visiblecontent element of that markup language document, which may be receivedvia user input as discussed above.

The process 400 begins in block 402, in which the selection facilityaccesses a hierarchical data structure containing information regardingcontent elements of the markup language document. The hierarchical datastructure may be any suitable hierarchical data structure, including onemaintained by the viewing application. For example, in some embodimentsthe hierarchical data structure may be a Document Object Model (DOM)containing information on content elements described by markup elementsof the markup language documents. A hierarchy of the DOM may match ahierarchy of the markup elements, with some content elements arranged as“children” of other content elements within the hierarchy of the DOMwhen the corresponding markup elements are nested inside one another.

In block 402, the selection facility accesses the hierarchical datastructure to identify the node in that structure that relates to thefirst visible content element that was identified as being of interestto the user (e.g., based on user input). After identifying the node inthat structure for the first visible content element, the selectionfacility may determine the nodes for a chain of parent content elementsof the first visible content element extending up the hierarchy. Theselection facility may identify all parent elements in a chain extendingall the way up the hierarchy, or may identify any suitable number ofparent elements that is less than all. In some embodiments, it may beadvantageous to identify only up to a certain number of parent elementsto limit the computation that is subsequently performed in the process400 and increase overall efficiency of the process 400. For ease ofdescription herein, in some cases the chain of parent content elementsmay be identified as those content elements that are “directly above”the first visible content element in the hierarchy, and it should beappreciated that this phrase does not refer to only content elements inthe next level immediately above the first visible content element inthe hierarchy.

In some embodiments, in addition to identifying parent elements in thechain, the selection facility may identify child elements in the chainas well, through identifying elements directly below the first visiblecontent element in the hierarchical data structure until the end of thestructure is reached.

The set of content elements identified from the hierarchical datastructure in block 402 is a set of candidate content elements (some ofwhich may not be visible content elements) that are to be evaluated todetermine whether they represent a set of related visible contentelements (including the first visible content element and any relatedcontent). Based on the evaluation, one of the candidate content elementsidentified in block 402 will be selected. If the selected candidatecontent element is the first visible content element (or is anothercandidate content element that is a container including only the firstvisible content element and no other visible content elements), thenthis represents a determination that there are no other visible contentelements that are to be selected along with the first visible contentelement and, for example, emphasized along with the first visiblecontent element by an emphasis facility. If, however, the selectedcandidate content element is a container including the first visiblecontent element and other visible content elements, then this representsa determination that the first visible content element is a member of aset of related content elements distinct from surrounding visiblecontent elements and that, for example, the related set of visiblecontent elements should be emphasized together as a group by an emphasisfacility.

Accordingly, in block 404 the selection facility begins an analysis ofeach of the candidate content elements identified in block 402. For eachcandidate content element, the selection facility in block 406determines information on the candidate content element. The informationthat is determined may include information on a type of the candidatecontent element, such as a type of the candidate content element and/ora type of markup element that defined the candidate content element.Information on a size of the candidate content element may also bedetermined, as well as information on a position at which the candidatecontent element is to be displayed within a display the markup languagedocument by a viewing application.

Information on a separation of the candidate content element fromsurrounding visible content elements may be also determined in block404. The separation may be determined as the amount of space betweenexpressive content elements, including any margins or padding on anexpressive content element set based on the formatting of the contentelement and on any other whitespace (which, as those skilled in the artwill appreciate, may or may not be “white”) that may exist between theexpressive content elements due to positioning of the elements. Theseparation may be determined for each of the four sides of the candidatecontent element on a two-dimensional screen.

In some embodiments, visual separation may be evaluated in connectionwith criteria related to a manner in which humans typically perceivegroups of content. For example, criteria may be developed based onpsychological research regarding human perception of groups of content,such as based on Gestalt Laws of Grouping. In connection with thesecriteria, a selection facility may evaluate distances separating visiblecontent elements and other factors that relate to perceived groupings ofcontent.

Separation may be determined for candidate content elements in block 404based on expressive content elements, whether or not the candidatecontent element is an expressive content element. Expressive contentelements may be text element, image element, or other type of visiblecontent elements that convey information to a viewer (or may convey,given that an image may, in some cases, merely be a solid color box), asopposed to content elements that are not visible or that are visible butmerely serve as layout elements such as borders or lines. In the casethat the candidate content element is an expressive content element, theseparation may be the distance on each side from the boundary of theexpressive content element to another expressive content element,without taking into account intervening non-expressive content elements.Alternatively, in the case that the candidate content element is not anexpressive content element, then the separation may be determined basedon expressive content elements contained within the non-expressivecontent element. For example, if the candidate content element is notitself an expressive content element but contains an assortment ofexpressive content elements, then the separation may be determined foreach of the four sides of the non-expressive content element based onthe expressive content element within the non-expressive content elementthat is closest to that side. More specifically, for the left side of anon-expressive content element, the separation may be determined byidentifying the expressive content element within that non-expressivecontent element that is closest to the left side, determining thatleft-hand separation for that expressive content element, and using thatdetermined separation as the left-hand separation for the non-expressivecontent element. A similar process may be repeated for the other threesides of the non-expressive content element.

While the information-gathering step of block 406 is shown in theexample of FIG. 4 as included within the loop of block 404 and carriedout for one candidate content element at a time, it should beappreciated that embodiments are not so limited. In some otherembodiments, the selection facility may gather the information of block406 for all candidate content elements at once, to efficiently gatherthe information once for all candidate content elements for subsequentanalysis.

In block 408, once the selection facility retrieves the information onthe currently-selected candidate content element of the loop of block404, the selection facility evaluates the information for the candidatecontent element in connection with a number of factors. The selectionfacility is not limited to evaluating any particular factors in itsanalysis and examples of factors are discussed in greater detail belowin connection with FIG. 5. The factors that may be analyzed may befactors that are indicative of whether the candidate content elementrepresents a distinct set of content that is distinct from surroundingcontent. Such a distinct set of content may be content that isconceptually related or that a publisher of the markup language documentotherwise intended a viewer to view together as a set. For example, aseparation for the candidate content element may be evaluated because agroup of content elements that are more separated from surroundingcontent elements may be more likely to be a distinct set or otherwiseintended to be viewed together. This is because it is common forpublishers of markup language documents to separate from surroundingcontent a group of content that should be viewed together. Similarly, alayout of each candidate content element may be evaluated to determinewhether it is visually separated from surrounding visible contentelements in a way that suggests it is distinct, such as by including adifferent background from surrounding visible content elements. Asanother example, a structure of markup elements defining the candidatecontent element and markup elements that are adjacent in the markuplanguage document may be evaluated to determine whether the markupelements indicate a repeating layout of cells in one or more columnsand/or one or more rows. In this case, the candidate content elementthat represents a single cell of that repeating layout may be best toselect as a distinct set of related content.

Each of the factors that is scored in block 408 may be associated with aweight. The weight may indicate a relative importance toward selection,with positive, higher-value weights indicating that a factor weighsheavily in favor of that candidate content element being selected. Someweights may be negative, where a factor influences the selectionfacility toward not selecting a candidate content element. Once thescores for a candidate content element are calculated in block 408, inblock 410 the selection facility weights the scores and, in block 412,processes the weighted scores to calculate an overall score for thecandidate content element. The selection facility may weight the scoresand process the weighted scores in blocks 410, 412 in any suitablemanner, as embodiments are not limited in this respect. In oneembodiment, the facility may multiple the scores by correspondingweights for factors and then sum the weighted scores. In otherembodiments, other mathematical calculations may be used. In still otherembodiments, a neural network of inputs may be used to calculate anoverall score based on scores for factors.

In block 414, the selection facility determines whether more candidatecontent elements are to be evaluated in the loop of block 404 and, ifso, continues through the loop with the next candidate. Once emphasisfacility determines in block 414 that the loop of block 404 has beenperformed for each candidate content element and the selection facilityhas calculated an overall score for each candidate content element, inblock 416 the selection facility chooses the candidate content elementhaving the highest score. As discussed above, the candidate contentelement may, in some cases, be the first visible content element thatwas identified (e.g., from user input) as being of interest to a user.The candidate content element may alternatively, in some cases, beanother content element that may be a container for the first visiblecontent element and potentially other, related visible content elementsof the markup language document. The selected candidate content elementrepresents an identification of the visible content elements selected bythe selection facility, as the visible content elements that areselected by the selection facility to be, for example, emphasized by anemphasis facility will be those visible content elements that aredisposed within the selected candidate content element in thehierarchical data structure that was accessed in block 402.

Once a candidate content element is selected in block 416, the process400 ends.

As discussed above in connection with block 408, the selection facilitymay calculate a number of scores for a candidate content element inconnection with a number of factors that may be evaluated. FIG. 5illustrates an example of a process 500 that may be used by theselection facility to implement the scoring of block 408 for eachcandidate content element within the context of the loop of block 404.It should be appreciated, however, that the example of FIG. 5 is merelyillustrative and that other processes and other factors may beevaluated.

The process 500 begins in block 502, in which the selection facilityevaluates a size of the candidate content element. The size that isevaluated may be both an absolute size and a relative size to a nextcandidate content element both up and down in the chain of candidatecontent elements of the hierarchy. An absolute size for the candidatecontent element may be the dimensions of the candidate content element,such as the width and height. The width and height may be the defaultwidth and height (e.g., in pixels) for the candidate content element, asset out by original markup elements of the markup language document, ormay be the width and height (e.g., in pixels) for the candidate contentelement as manipulated by the viewing application, such as if theviewing application has increased the size of the entire markup languagedocument and, with it, the size of the candidate content element.

The selection facility may evaluate the absolute size in any suitablemanner. For example, the selection facility may compare the absoluteheight and/or the absolute width to thresholds to determine if eitherthe height and/or width are below minimum thresholds and/or abovemaximum thresholds. If the height and/or width are below minimumthreshold or above maximum thresholds, then corresponding size factorsmay be set to high values, which may then be given negative weights in asubsequent step, to give an effect toward not selecting the candidatecontent element. There may be thresholds for all content, as well as aspecial threshold for very wide or very tall images. Wide or tall imagesmay be design elements that, in some embodiments, should never beselected. Standard thresholds for other elements may influence aselection, but leave open the possibility of a candidate content elementhaving sizes above the maximum thresholds or below the minimumthresholds being selected. A significantly high value or a significantlyhigh negative weight may be used to ensure that very wide or very tallimages are not selected by the selection facility.

A related analysis may be whether the height and/or width fall withinranges of dimensions that have been identified to be “good” dimensionsfor potential sets of related content elements. For example, a range of“ideal width” for a candidate content element may be determined to bebetween 20 percent and 85 percent of the width of the display area ofthe viewing application (e.g., viewport of a web browser) and a range of“ideal height” for a candidate content element may be determined to bebetween 25 percent and 60% of the height of the display area. In thecase that the height and/or width of the candidate content element fallswithin the ideal ranges, high values may be given to one or both ofthese scores, and positive weights may be associated with the factors,to influence the selection facility toward selecting the candidatecontent element. In the case that a height and/or width falls outside ofthe ideal range, scores for factors of “percentHeightOverIdealMax”,“percentWidthUnderIdealMin,” and “percentWidthOverIdealMax” arecalculated as the difference between the percentages of the height/widthof the candidate content element relative to the display area of theviewing application and the ideal height/width percentages given above.

In the case that a criteria for one of the size factors is not met, orany other criteria for other factors described below, is not met and isthus inapplicable to a candidate content element, the factor is assigneda score of 0.

In addition to evaluating an absolute size of the candidate contentelement in block 502, a relative size of the candidate content elementmay also be evaluated. The relative size that is evaluated may be therelative size between the candidate content element and a parent or achild content element (for which information may be available becausethese elements are also in the chain and being analyzed as candidatecontent elements). The relative size may measure growth of the candidatecontent element versus the parent or child element. Some types of growthmay be good indicators that the candidate may be better to select than aparent or a child, while other types of growth may indicate that thecandidate is worse. For example, if the candidate content element growssignificantly in two dimensions relative to the immediate child, thenthis is a negative with respect to identifying a group of relatedcontent elements, as groups of related content elements typically growonly in one dimension. Significant growth in two dimensions may moreclearly signify that a candidate content element is a group of groups ofrelated content, and it would be preferable for the selection facilityto select one of those groups instead. Significant growth in this casemay be measured versus a threshold, such as more than 30 percent growthhorizontally or more than 25 percent growth vertically. In the case thatthe two thresholds are met, a high score may be assigned to the factorand a high negative weight may be associated.

Large growth (e.g., over the thresholds specified above) in only onedirection (horizontal or vertical) may also be disadvantageous, inaddition. For example, if a candidate content element is much wider thanits immediate child, this may be another sign that the candidate contentelement is a row of groups of content, and it would be preferable forthe selection facility to pick one of the groups. Accordingly, a highscore will be assigned to this factor with a negative weight.

In some cases, an amount of growth for a candidate content element overa child may be important based on dimensions of the immediate child. Forexample, if the immediate child is tall or wide, which may be judged bya height or width above thresholds, growing any amount taller or widerthan the already tall/wide child may be undesirable as it may notindicate a group of content that should be selected. In this case, ahigh value with a high negative weight may be used for this factor.

Moderate growth in only one direction, however, such as growth lowerthan these thresholds, may be a very good sign. For example, a containerfor a picture and a caption, or a paragraph and a title may bemoderately larger in a vertical direction than the picture or than theparagraph, and it would be best for the container to be selected ineither case. Accordingly, in this situation, a high score may beassigned to the factor and a high weight associated. Similarly, if acandidate content element is moderately smaller than a parent in onedirection, to encourage the parent that is moderately larger to bechosen the candidate content element will be assigned a high score for a“moderately smaller” factor that has a high negative weight associated.

In block 504, the selection facility may also evaluate the separation ofthe candidate content element from surrounding visible content elements.The separation may be judged on whitespace, the existence of borders, orother elements of a layout that serve to visually separate the candidatecontent element from surrounding visible content elements. For example,the separation information relating to space that is collected for acandidate content element (as discussed above in connection with block406 of FIG. 4) may be used to determine a score for top separation,bottom separation, left separation, and right separation that has ascore relative to the separation value calculated for each side. As analternative, a vertical separation or horizontal separation may becalculated for each candidate content element based on the separationvalues for top and bottom (for vertical) and left and right (forhorizontal) and on whether the candidate content element is adjacent toa border on the top and/or bottom (for vertical) and adjacent to aborder on the left and/or right (for horizontal). In some embodiments,to increase the separation score as the amount of whitespace increases,the amount of whitespace in pixels may be raised to the power of 1.3before being scored.

Separation may also be evaluated by the presence of borders. Regardlessof the separation values based on whitespace, if the candidate contentelement is adjacent to a border the corresponding separation value maybe high, as the border creates a stronger visual separation than anyamount of whitespace could. In the case of a border, the separationscore may be calculated based on the width of the border, such as ascore of 10 points per pixel of border width. A border may be detectedbased on the formatting of the candidate content element, such as wherethe style information for the candidate content element indicates aborder/line is to be applied when the candidate content element isrendered. The border may also be created as a thin image, such as animage that is tall and thin for a vertical border or wide and short fora horizontal border. As another example, the border may be created as a“horizontal rule” element specified by an “hr” HTML tag.

In block 504, the selection facility may also judge a visual separationof the candidate content element based on the background of thecandidate content element and whether the background differs from thebackground of the next parent element of the candidate content element.If the background differs from the parent, this is a strong indicationthat there is a distinct set of content, as it is common for webdevelopers or other publishers of markup language documents to aggregatedistinct sets of content within boxes that would appear as differentbackgrounds during the automated analysis by the selection facility. Inthis case, the “has different background” factor would be assigned ahigh score, and would have a high positive weight.

In block 506, the selection facility may evaluate contents of thecandidate content element to determine whether there are contentsindicative of the presence of multiple different groups of contentwithin the candidate content element. For example, if there are elementshaving different background colors within the candidate content element,this is indicative of different groups. Similarly, the existence ofborders and lines such as thin images, horizontal rules, or borders andlines established by style of a candidate content element is alsoindicative of the presence of multiple groups. The existence of multiplesection headers, as indicated by multiple elements specified using theHTML tags “h1” through “h6” may also be indicative of the presence ofmultiple items.

In block 508, the selection facility may evaluate whether the candidatecontent element (and any descendent content elements, in thehierarchical arrangement of content elements) includes only a singleheader (as specific by HTML tags “h1” through “h6”) potentially coupledwith other elements or includes only a single image potentially togetherwith other content. It is common for web content to be arranged as asingle image alone or coupled with related text (e.g., a caption or textotherwise explaining the image) or as a single section header alone orcoupled with related text or images. In these cases, the presence of asingle header or a single image within the candidate content element(and its descendants, if any) is a strong indication that the candidatecontent element defines a set of related content. Accordingly, in eithercase the related factor would be assigned a high value and positiveweight would be associated.

In block 510, the selection facility may evaluate a type of thecandidate content element to determine whether the candidate contentelement is a “great” element type, a “good” element type, or a “bad”element type, as indicated by the HTML tag that defines that defines thecandidate content element. As indicated by the names, a “great” elementtype may be given a high value or a high weight, a “good” element may begiven a moderately high value or high weight, and a “bad” element typemay be given a high value or negative weight. Great elements may includethose that have been identified as commonly being used by web developersto identify discrete groups of related content (or content that is alonebut should be picked alone rather than as a part of a group). Forexample, block quotation elements (“blockquote”), a cell of a table(“td”), an unordered list of content (“ul”), an ordered list of content(“ol”), or a menu (“menu”) may be commonly used by web developers toidentify discrete groups of related content and may therefore be greattags. Good elements may be often, but not always, used in ways toidentify groups of related content (or content that is alone but shouldbe picked alone rather than as a part of a group). A link (“a”) thatincludes a block of content, a street address (“address”), a button(“button”), a snippet of computer code (“code”), a description list(“dl”), a group of related form controls (“fieldset”), a form as a whole(“form”), a paragraph (“p”), a group of preformatted content (“pre”), alist item (“li”), a section (“section”), a table row (“tr”), or any oneof the types of controls of a form may be good tags. Examples of badelements that are not often used for groups of related content or fordiscrete content elements include any of the HTML header tags (“h1”through “h6”) or a group of headers (“hgroup”). The determination ofwhether the candidate content element is of a bad type may thereforealso be a determination of whether the candidate content element is aheading.

In addition to evaluating the type of the candidate content element inblock 510, a type of ancestor (parent, grandparent, etc.) elements mayalso be evaluated. Some types of content elements may not commonly beused as containers of groups of content elements, and thus it should beuncommon for a descendant (child, grandchild, etc.) of one of thesetypes of elements to be selected. For example, if any one of theancestor elements of the candidate content element is a list item(“li”), a paragraph (“p”), a header (either “h1” through “h6” or“hgroup”), or is a button (“button”), then the candidate content elementshould not be selected.

In block 512, the selection facility evaluates the candidate contentelement to determine whether the layout and/or markup elements of thecandidate content element and surrounding content indicate that it is acell of a repeating layout (e.g., a row, column, or grid of repeatingcells). It should be appreciated that, in the context of a repeatinglayout, a “cell” may be any suitable grouping of content that isrepeated, and is not limited to be arranged in a table format or usingmarkup elements related to tables. Any suitable layout or format forrepetition may be used and may be analyzed in block 512.

In the case that the markup language document includes a repeatinglayout of cells, it is often desirable to select one of the cells as thediscrete set of related content elements, as web publishers often placerelated content within a cell and unrelated content in different cells.Users often desire to view an entire cell at one time, as well.Accordingly, in block 512 the selection facility determines whether thecandidate content element is of a type that is not commonly associatedwith cell layouts, such as a paragraph (“p”), link (“a”), or heading(“h1” through “h6”). The selection facility may also determine a numberof sibling elements for the candidate content element by accessing thehierarchical data structure describing content elements of the markuplanguage document (e.g., DOM). If the hierarchical data structureindicates that the candidate content element is alone on its level ofthe hierarchy and has no siblings, there is thus no repeating layout atthat level and the candidate content element cannot be determined to bea cell of a repeating layout. For these two factors, a high value and anegative weight may be used. The selection facility may also determinewhether a height of the candidate content element is below a thresholdassociated with cells, such as 25 pixels (e.g., because repeating cellsseldom have heights below this threshold). This may be associated with ahigh value and a negative weight. The selection facility may thendetermine whether, if the candidate content element has one sibling, ifthe candidate content element is sharing a horizontal or verticaldimension of the parent element roughly evenly with the other sibling.For example, the selection facility may determine whether the width orheight of the candidate content element is between 33 percent and 66percent of the width or height of the parent. If the two siblings arenot sharing the height or width roughly evenly, then it is unlikely thatthe two elements are cells of a repeating layout. In this case, a highvalue and a negative weight may indicate the unlikelihood.

If, however, the selection facility finds more than two siblings or twosiblings that are sharing a parent's dimensions roughly evenly, then theselection facility may evaluate whether the siblings are arranged in away that is indicative of them being cells in a repeating layout. Theselection facility may determine whether the cells include largely thesame tags. If the siblings are not largely the same in content andlayout, then it is unlikely that the siblings are part of a repeatinglayout. In this case, the selection facility may not look for thesiblings to be identical, but may look for a threshold level ofsimilarity, such as having an overlap of tags between siblings of 80percent or more. Sharing 80 percent of their tags or more may be a goodsign and be given a high value and a positive weight. The dimensions ofthe siblings may also be evaluated to determine whether the dimensionsof the siblings tend to vary in only one dimension. It is common forcells of a repeating layout to vary in one dimensions, such as width orheight, but uncommon for cells to vary in two dimensions. Varying inonly one dimension may be assigned a high value and a positive weight,while varying in two dimensions may be assigned a high value and anegative weight. Having precisely matching dimensions, such as exactwidth or exact height, between siblings may be a strong indicator andassigned a high value and a strong positive weight.

When comparing siblings in the manner described in the precedingparagraph, it may increase efficiency to examine only some siblings,rather than all, as a great number of siblings may increase thecomputational complexity of the analysis and take a great deal of time.In some embodiments, then, only up to a threshold number of siblings maybe evaluated, such as up to five siblings.

Relative dimensions to a parent content element may also be evaluated.If a parent content element has a same width but significantly greaterheight than a candidate content element, this suggests that thecandidate content element may be a cell in a repeating column of cells.Similarly, a parent with an exactly matching height and a significantlygreater width may indicate that the candidate content element is a cellin a repeating row of cells.

Web developers may commonly use elements that are set to be “floats” fortwo reasons: first, to flow text around image elements or other contentgroups, and second, to create cells of repeating layouts. When acandidate content element is configured as a “float,” the selectionfacility may determine whether that candidate content element is alsoconfigured for text to wrap around it. If it is configured for textwrapping, it is more likely that the float is to be used for textwrapping and less likely that it is a sign of a cell of a repeatinglayout. If, however, it is a float that is not configured for textwrapping, this may be a sign that it is a float that is being used todefine a cell of a repeating layout. This may be associated with a highvalue and a strong positive weight in this case.

In block 514, the selection facility may also evaluate whether thecandidate content element is a list of elements, including a menuelement such as a menu of links. The selection facility may beconfigured not to select a horizontal list, including a horizontal menubar, as it is unlikely that a user will want to emphasize an entirehorizontal menu bar together or otherwise select an entire horizontalmenu for performance of a task. Accordingly, a list of elementspositioned horizontally, especially a wide horizontal list, may beassigned a high value with a negative weight. A vertical list, however,may be assigned a high value as it may be more likely that a user willwant to view a vertical list of elements together. A high value with apositive weight may be assigned to containers of lists that are arrangedvertically. If each element on the list is a link, then it may be evenmore desirable to select together, as a user may want to view/emphasizeor otherwise process the links together and potentially select one ofthe links. Accordingly, when the vertical list is a list of linkelements (“a” elements), the score determined for the list may bemultiplied by a value (e.g., 1.5) to give a bonus score for being a listof links.

After completing the menu or list analysis of block 514, the process 500ends. As a result of the process 500, a set of scores has beencalculated for various factors by which a candidate content element isevaluated. The scores may be combined according to weights to yield anoverall score for the candidate content element. The process 500 mayalso be repeated for each candidate content element of a set ofcandidate content elements and, as discussed above in connection withFIG. 4, the candidate content element with the highest score may beselected.

It should be appreciated, as discussed above, that while FIG. 5illustrates a set of factors that may be evaluated, the factors aremerely illustrative. Other factors may be considered, and the factorsmay be combined in any suitable combination, as none of the factors arenecessary. Embodiments are not limited to carrying out any particularanalysis of factors for candidate content elements.

In some embodiments, once the factors are scored as in the example ofFIG. 5, the scores may simply be multiplied by corresponding weights andsummed to calculate an overall score that is used to determine whether acandidate content element is selected by the selection facility. Inother embodiments, however, the selection facility may adjust the scoresbetween parent/child candidate content elements as part of selecting acandidate content element, and may also conduct a more detailed analysisof candidate content elements when scores for two candidate contentelements are close. Both such processes may be carried out to “smooth”results between different iterations of the analysis process andencourage consistent results.

Process 600 is an example of a process that may be carried out as partof smoothing results. Prior to the start of process 600, a process maybe performed for evaluating candidate content elements under a varietyof criteria, from which a set of overall scores may be calculated. Theprocess 600 begins in block 602, in which a share of a child candidatecontent element is transferred to its immediate parent candidate contentelement when one or more criteria are met. Any suitable criteria may beused, including criteria that, when met, indicate that a parent shouldbe preferred for selection over its child. For example, if a parentcandidate content element has only one child, it is likely that theparent should be selected over the child and, as such, a portion of thechild's score may be transferred to the parent to encourage theselection facility to select the parent before the child (though a thirdelement with a still-higher score may still be selected by the selectionfacility, as that relationship is unchanged). For example, 75 percent ofthe child's score may be subtracted from the child's score and added tothe parent's score. As another example of a criterion, when dimensionsof a parent candidate content element are only slightly larger than itsimmediate child, it is again unlikely that the selection facility shouldcorrectly select the child element over the parent. To account for thisunlikelihood, some portion (e.g., 30 percent) of the child's score maybe subtracted from the child's score and added to the parent's score. Asa third example, if the parent candidate content element contains only aheading and one other content element (either of which may be the childcandidate content element), it is more likely that a correct selectionbetween the parent and the child should be of the parent. Accordingly, aportion (e.g., 75 percent) of the child's score may be subtracted fromthe child's score and added to the parent's score.

In block 604, the selection facility may compare overall scores for eachof the candidate content elements, some of which may have been adjustedin block 602. The selection facility determines in block 604 whether asecond-highest score is within a threshold amount of the highest score.The threshold amount may be an absolute number, such as a number ofpoints, or may a relative number, such as a percentage. If thesecond-highest score is not within the threshold amount, then in block606 the selection facility selects the candidate content element havingthe highest score. If, however, the second-highest score is within thethreshold amount of the highest score, then the selection facilitycarries out a tie-breaking process. In the tie-breaking process, inblock 608 the selection facility determines which, between thehighest-scoring candidate content element and the second-highest-scoringcandidate content element, has the larger dimensions. For the largerelement, in block 610 the selection facility then identifies as a newset of candidate content elements all of the expressive content elementsthat are contained within the larger element according to thehierarchical data structure of content elements (e.g., DOM) and thathave no child content elements themselves (e.g., are “leaf” elements ofthe hierarchical data structure). The selection facility may thenevaluate each of these expressive content elements within the largerelement and the larger element itself using the same evaluation andscoring process previously used for the visible content elementindicated by user input. For example, the process 400 of FIG. 4,including the accessing of the hierarchical data structure to identifyancestor content elements as in block 402 and the scoring of eachancestor element, may be performed for each of the expressive “leaf”content elements to determine new scores. For each “leaf” element, thehighest scoring candidate content item is identified and evaluated todetermine whether it is the candidate content element determined inblock 608 to be the larger element. In particular, the selectionfacility in block 612 determines whether the newly-identifiedhighest-scoring element for each of the “leaf” content elements is thelarger element identified in block 608 and whether more than a fraction(e.g., 50 percent) of the highest-scoring element for the “leaf” contentelements is the larger element. If more than the fraction ofhighest-scoring elements for the “leaf” elements are the larger elementof block 608, then in block 614 the candidate content element determinedin block 608 to be the larger element is selected by the selectionfacility. If, however, the selection facility determines in block 612that less than the fraction of highest-scoring elements for the “leaf”elements are the larger element, then in block 616 the candidate contentelement determined in block 608 to be the smaller element is selected bythe selection facility. Once one of the candidate content elements isselected in blocks 606, 614, or 616, the process 600 ends.

While the transfer of scores between parent and child elements and thetie-breaking process were described above as being implemented together,it should be appreciated that embodiments are not so limited. In someembodiments, the parent-child score transfer may be implemented withoutthe tie-breaking process or with a different tie-breaking process. Inother embodiments, the tie-breaking process may be implemented without aparent-child score transfer or with a different score transfer process.

In some embodiments that implement a selection process according to someor all of the techniques described in connection with FIGS. 3-6, asingle element is selected as the output of each selection process,which represents a set of one or more visible content elements containedwithin that single element. In some such embodiments, to ensure that asingle element is selected, a tie-breaking process such as thetie-breaking process of FIG. 6 may be implemented. It should beappreciated, however, that embodiments are not limited to selecting onlyone element as the output of the selection process. In some embodiments,one, two, or more elements may be selected. For example, if two or moreelements have the same highest score, or have the highest scores andhave differing scores within a threshold difference, the multipleelements may be selected. In this case, all of the visible contentelements below each of the selected elements may be included in the setof related visible content elements.

Those skilled in the art will appreciate that the selection techniquesdescribed above may in some cases be limited and not work well for somemarkup language documents. The analysis techniques described aboveinclude determinations made based on an arrangement of markup elementsin the markup language document that follows common arrangements that amarkup language document may be expected to follow. For example, oneexpected arrangement on which some techniques described above are basedis that related content elements are disposed within a same portion of ahierarchy of content elements for the markup language document, whichmay also be termed being part of the same “subtree” of the hierarchy.While many publishers of markup language documents may arrange markupelements for content elements in this manner, not all publishers followthis practice. Accordingly, in some cases, related content elements mayhave markup elements arranged in different subtrees of the hierarchy.

In some embodiments, a selection facility may account for thepossibility of related content being arranged in different subtrees byexamining a layout of content elements in an area of the first visiblecontent element, that was identified by user input as the contentelement of interest, to determine whether adjacent content elements meetcriteria for being potentially related. For example, if the firstvisible content element (that was indicated by user input) is a formcontrol, the selection facility may examine the adjacent visible contentelement (if any) to the left above the form control and the adjacentvisible content element (if any) above the form control to determinewhether the either adjacent visible content element is within athreshold distance and is a text element containing a small amount oftext (e.g., up to a threshold number of words or characters). If eitheradjacent visible content element is a short text that is close to theform control, then it is possible that the adjacent visible contentelement is a label for the form control that should be selected by theselection facility as related to the form control. As another example,if the first visible content element is a paragraph, the selectionfacility may examine up to a threshold number of visible contentelements disposed vertically above and vertically below the firstvisible content element to determine whether, for example, the paragraphappears in a collection of paragraphs that is bounded above by a headingor whether the paragraph appears in a collection of paragraph boundedabove and below by headings. In either case, the paragraphs and theheading at the top are likely to be related content and the selectionfacility may select the content.

Accordingly, in some embodiments the selection facility may examine alayout in an area of the markup language document in which the firstvisible content element (indicated by user input) is displayed toevaluate whether adjacent visible content elements satisfy criteria forbeing related. The criteria may be based on a type of the first visiblecontent element. The adjacent content elements that are evaluated mayinclude immediately adjacent visible content elements or visible contentelements that are up to a threshold number of content elements away.

Through such a selection process, the selection facility may identify asrelated visible content elements that are disposed in multiple differentsubtrees of the markup language document. The selection facility maythen simplify its selection by identifying whether any of the selectedvisible content elements are containers of other selected visiblecontent elements, such that the container elements would be parents (orgrandparents, etc.) of other selected visible content elements in thehierarchy of content elements of the markup language document. If so,then the selection facility may remove from its selection the child (orgrandchild, etc.) elements, as the selection facility will in a laterstep select all child elements of the selected elements, as should beappreciated from the foregoing and discussed again below. Once theselection facility has simplified its selection in this way, theselection facility will have selected one or more content elements thatare each a top-level element with respect to one another, in that noelement is a container for any of the other elements. From there, theselection facility will select as the set of related visible contentelements all visible content elements included below those one or moretop-level elements.

As a result of one of the processes of FIG. 3, 4, or 6, a selectionfacility may select a single visible content element of a markuplanguage document or a group of visible content elements that theselection facility has determined may be conceptually related, or areotherwise a distinct set of visible content elements. In someembodiments, the visible content elements selected in this manner maythen be identified to an emphasis facility, as discussed above and inmore detail below, and the markup language document may be edited toemphasize these visible content elements together in context with oneanother. It should be appreciated, however, that other embodiments arenot so limited. The selection process described herein may be used forpurposes other than emphasis and embodiments are not limited to usingthe selected set of visible content elements for any specific reason. Asanother example, the selection process might be used by annotationsystem in which a user inserts notes or other annotations relating tocontent of a markup language document. For example, a user may indicateone visible content element and the system may identify, using theselection techniques described above, set of related visible contentelements, and the annotation may be applied to the set of relatedvisible content elements as a group. Any type of task may be performedon visible content elements selected using the selection techniquesdescribed herein, as embodiments are not limited in this respect.

In some embodiments, though, once the selection facility has selected aset of one or more related visible content elements, the selectionfacility may identify that set of one or more related visible contentelements to an emphasis facility. The emphasis facility may then editthe markup language document to change a manner of display of the one ormore visible content elements, such as in some cases to increase avisibility of the one or more visible content elements. It should beappreciated that embodiments are not limited to changing the manner ofdisplay of visible content elements in any particular manner. FIGS. 7-9illustrate examples of techniques that may be implemented by an emphasisfacility in some embodiments. For ease of description below, the set ofvisible content elements will be described as a plural set, rather thancontinuously as a “set of one or more.” It should be appreciated,however, that in some cases the selection facility may identify only onevisible content element for which there are not related visible contentelements together with which it is to be emphasized.

FIG. 7 illustrates an example of a process by which an emphasis facilitymay respond to user input by modifying a markup language document toemphasize visible content elements. The process 700 may be carried outafter a viewing application has downloaded and processed a markuplanguage document, into which the emphasis facility may be incorporated.The process 700 may also be carried out after the visible contentelements of the markup language document have been displayed. In someembodiments, as should be appreciated from the discussion of process 700below, a selection process carried out by a selection facility may alsobe performed prior to the start of the process 700 to identify a set ofrelated visible content elements.

The process 700 begins in block 702, in which a first user input isreceived. The first user input may be a user input requesting that afirst change to a manner of display of the set of visible contentelements be applied. In some embodiments, the first user input may be auser input specifically associated with a request that the first changebe made, such as a keystroke specifically associated with the firstchange or a selection of a user interface button associated with thefirst change. In such cases, the set of visible content elements towhich the first change is to be made may have been identified before thestart of process 700, such as based on an earlier user input identifyingone visible content element. In other cases, though, the user inputreceived in block 702 may be a user input that indicates a visiblecontent element, and the selection facility may identify the set ofvisible content elements based on that same user input. Followingreceipt of the user input in block 702 (and, in some embodiments,identification of the set of related visible content elements), theemphasis facility edits the markup language document in block 704 tomake a first change to the manner of display of the set of relatedvisible content elements. In block 706, the emphasis facility receives asecond user input, which may be a user input specifically requestingthat another change to the manner of display be made. In block 708, inresponse to the second user input, the emphasis facility again edits themarkup language document to change the manner of display of the set ofrelated visible content elements. The editing of blocks 704, 708 may besimilar, in that edits may be made to a set of markup elements relatedto the set of related visible content elements. The editing may involvemodifying markup elements that were in the original, unedited markuplanguage document and/or inserting new markup elements, and the editingin block 708 may involve editing markup elements that were previouslymodified in block 704 or that were inserted in block 704. In block 710,the emphasis facility receives a third user input and, in response tothe third user input, in block 712 reverses the editing of blocks 704,708 to restore the set of related content elements to their originalform, without emphasis. In some embodiments, the reversing of block 712may be done in two stages in response to two user inputs, following thetwo-stage form of the editing of blocks 704, 708. For example, inresponse to a third user input, the emphasis facility may undo theediting of block 708, and in response to a fourth user input, theemphasis facility may undo the editing of block 704. As should beappreciated from the foregoing discussion of FIG. 2, the emphasisfacility may implement the undoing by storing an original, unedited formof the markup elements for the set of related visible content elements.To carry out the two-stage reversal, the emphasis facility may similarlystore information regarding the markup elements following the editing ofblock 704 and prior to the editing of block 708. Once the editing isreversed in block 712, the process 700 ends.

It should be appreciated that the user input received in blocks 702,706, 710 may be any suitable form of user input. Examples of user input,including via pointing devices, keyboards, and touch interfaces, aredescribed above in connection with FIG. 2. Any of these types of userinput, or other types of user input, may be implemented in embodiments,as embodiments are not limited to operating with any particular form ofuser input. In some embodiments, the user input of block 702 may be aninput moving a cursor of a pointing device to hover over a particularvisible content element that both indicates to the selection facility avisible content element of interest to the user from which a selectionof related visible content elements can be made and requests that theemphasis facility make first change to the display, and the user inputof block 706 may be a keystroke, such as a keyboard input, that requeststhat the emphasis facility make the second change to the display.

It should also be appreciated that the emphasis facility is not limitedto making any particular changes to the display of the set of relatedvisible content elements. In some embodiments, the first change that ismade may be to highlight the set of visible content elements and thesecond change that is made may be the magnify the set of visible contentelements while keeping other visible content elements the same size.However, other embodiments may combine either or both of these changeswith other changes, or make may different changes. In some embodiments,for example, font properties of text may be changed by an emphasisfacility. The alteration of font properties may include changing atypeface of text to a typeface preconfigured by a user, such as one theuser finds easy to read, or preconfigured by the emphasis facility asone generally easier to read such as a non-serif typeface or a typefacethat is designed for easy reading by users with dyslexia. Alteration offont properties may additionally or alternatively include changinginter-paragraph, inter-line, inter-word, or inter-character spacing,such as to provide more separation and make the text easier to read forusers with vision or reading difficulties. Alteration of font propertiesmay also include removal of or addition of italics, capitalization, orboldface to make the text easier to read. For example, the first letterof each sentence may be made boldface to make sentence separationseasier to see for some users. Alteration of font properties may alsoinclude removal or changing of the justification of text. For example,some users may find center-justified or right-justified text hard toread, or find the spacing variation in full-justified text hard to read,and the justification of the text may be changed to left-justified.Sentence layout properties may also be changed in text, such as byincreasing the spacing between sentences to make the separation betweensentences easier to perceive. Color properties of visible contentelements may also be adjusted. For example, bright colors may bedifficult for users to perceive content in those bright colors, and thecolors may be changed to be more muted colors that may be easier to see.As another example, where a background color of the visible contentelements provides low contrast with the foreground color of the visiblecontent elements, this may make viewing the content more difficult andthe emphasis facility may change one or both of the background color andforeground color to increase the contrast. Highlighting may also beinserted by the emphasis facility, such as by changing a backgroundcolor of the set of visible content elements and/or adding a border. Asize of the visible content elements may also be adjusted to be largerthan a default size.

Any of these changes, or other suitable changes, may be made to the setof related visible content elements. The changes may serve to improvevisibility of the set of related visible content elements. The changesmay also serve to visually separate the set of related visible contentelements from surrounding visible content elements, as the changes willnot be made to the surrounding visible content elements.

In some embodiments, in addition to or as an alternative to editing themarkup language document to improve visibility of the set of relatedvisibility content elements, the emphasis facility may also provide thecontent to another module for output in another medium, such as atext-to-speech program or a Braille display. In some such embodiments,the emphasis facility may cooperate with the other module to edit thedisplay along with the output via the other medium. For example, in oneembodiment the emphasis facility may emphasize each word of text in turnas that word being spoken by a text-to-speech program.

Accordingly, it should be appreciated that a variety of changes may bemade to a markup language document to emphasize visible content element,as embodiments are not limited in this respect. In connection with FIGS.8A-9, two examples of changes that are highlighting and magnificationare discussed, but embodiments are not limited to implementing theseparticular changes or any other changes.

In some embodiments, the emphasis facility may edit a markup languagedocument, once a set of related visible content elements have beenidentified, to visually separate those visible content elements fromsurrounding visible content elements using highlighting and a border.There are a variety of ways in which highlighting and borders may beinserted into markup language documents. FIG. 8A illustrates an exampleof a process for inserting highlighting and a border using vectorgraphics, with FIGS. 8B and 8C illustrating the form of the highlightingand an example of the highlighting applied to content, respectively.

A Scalable Vector Graphic (SVG) is a type of image that is defined usingmarkup language, specifically Extensible Markup Language (XML). As a“vector” graphic, the image is not defined as a series of pixels orpoints, but is instead defined by regions of color. For example, themarkup language content of an SVG may indicate a set of coordinates ordimensions for a polygon and indicate that the polygon should be coloredred. The inventor has recognized and appreciated that such an image maybe quickly defined by an emphasis facility by outputting dimensions forone or more polygons within an SVG. Moreover, the inventor hasrecognized and appreciated that such an image may be easily specifiedfor use within a markup element, such as a CSS markup element, byoutputting the markup for the SVG in a “Data URL” format in which themarkup language is listed in a URL in the same manner as elements of aweb address.

In some embodiments, an emphasis facility may add highlighting and aborder to a set of related visible content elements by using acombination of two or three vector graphics specified using markuplanguage, which in some cases may be SVG images. FIG. 8A illustrates anexample of such a process. The process 800 of FIG. 8A begins in block802, in which the emphasis facility determines dimensions of a boundingarea for the set of related visible content elements. The bounding areamay be a display area in which all of the visible content elements ofthe set are displayed. The emphasis facility may determine the boundingarea by determining the bounding area for each of the individual visiblecontent elements, and determining a rectangular area that extends overat least the union of the bounding areas for each of the individualvisible content elements. The bounding areas for each individual visiblecontent element may be determined using known techniques, includingJavaScript functions like “element.getBoundingClientRect( )” for imagesor other non-text objects, and for text objects by adding a DOM objectthat includes only the text and then using a“range.getBoundingClientRect( )” function on the new object. Whendetermining the bounding area for each of the individual visible contentelements, in some embodiments a padding or margin that is defined byformatting for a visible content element may be determined andsubtracted from the bounding area for that content element, to obtainthe true dimensions for the element. In addition, in some embodiments,the ancestor elements of the hierarchy for a content element may beevaluated to determine whether they apply any clipping that wouldprevent some of the bounding area of the content element from beingdisplayed. If so, then to obtain the true bounding area the clipping ofthe ancestor element may be applied to remove some of the bounding areaof the visible content element.

Once the bounding area for all of the visible content elements of theset of related visible content elements has been identified as therectangular union of the bounding areas of each visible content element,the bounding area is evaluated in block 804. Specifically, the boundingarea is evaluated in block 804 to determine whether the bounding areaincludes a part or a whole of any other visible content element. Forexample, unrelated content elements, such as floats, may extend over apart of the rectangle of the bounding box. In this case, to ensure thehighlighting is not incorrectly applied to these other visible contentelements, a shape of the bounding box should be changed from a rectangleto a polygon that wraps around but does not cover the other contentelements. To make the determination in block 804, each of the fourcorners of the rectangular bounding area identified in block 802 areevaluated to determine whether any of the four corners is a point atwhich is located a visible content element that is not one of the set ofrelated visible content elements, not a child (or grandchild, etc.) ofone of the set of related visible content elements, and not a parent (orgrandparent, etc.) of one of the set of related visible contentelements. If, in block 804, one or more of the corners is found to coveranother content element, then in block 806 the bounding area determinedin block 802 is edited. The emphasis facility may edit the shape anddimensions of the bounding area in block 806 to wrap around theelement(s) found at the original four corners of the rectangularbounding area, by determining the bounding areas of the found element(s)and adjusting based on the bounding areas of the found element(s). Insome embodiments, the adjustment of block 806 may only be performed forelement(s) identified in block 804 that have a float property set thatconfigures surrounding content to wrap around the found element.

Once the dimensions and shape of the bounding area have been edited inblock 806 (or was determined in block 804 not to overlap otherelements), the emphasis facility in block 808 creates a vector graphic,using markup language, that includes a polygon having the shape anddimensions of the bounding area. The emphasis facility may specify acolor of the polygon that will be the background highlighting color ofthe set of related visible content elements when the vector graphic isinserted into the markup language document. By having a polygon that hasthe same dimensions and shape as the arrangement of visible contentelements in the set, the emphasis facility can insert the vector graphicto highlight only the set of related visible content elements withouthighlighting some or all of any surrounding visible content elements,and thus visually separate the set of visible content elements.

In some cases, it may be disadvantageous to have highlighting thatextends only to the very edge of the set of visible content elements,and may instead be preferable for the highlighting to extend slightlybeyond. This may be because, in some cases, highlighting or a borderdisposed at the exact edge of the set of visible content elements mayobscure the edges of the visible content elements, such as making textmore difficult to read. Accordingly, in block 810 the emphasis facilityslightly expands the dimensions and shape determined in block 806. Theslightly expanded dimensions may cover the bounding box of the set ofrelated visible content elements plus a margin area. The emphasisfacility may then specify a second vector graphic, using markuplanguage, that includes a line following the expanded dimensions andshape and having a certain width, where the width corresponds to thedesired width of the highlighting beyond the edge of the visible contentelements. The emphasis facility may specify that the line have the samecolor as the polygon of block 808, such that the line and the polygonmay appear to blend together on a display screen as one highlightingelement.

In some embodiments, it may also be desirable to insert a border aroundthe highlighting, to further visually separate the set of relatedvisible content elements from surrounding visible content elements.Accordingly, in block 812 the emphasis facility again slightly expandsthe dimensions and shape determined in block 806. The slightly expandeddimensions may cover the bounding box of the set of related visiblecontent elements plus a second margin area that is larger than themargin area used in block 810. The emphasis facility may then specify athird vector graphic, using markup language, that includes a linefollowing the second expanded dimensions and shape and having a certainwidth, where the width corresponds to the desired width of the border.The emphasis facility may specify that the line have a different colorthan the color specified in blocks 808, 810, such as a second colorhaving high contract with the first color of blocks 808, 810 to providea definite border. In block 812, the emphasis facility may also edit theshape and dimensions (after expanding them) to insert rounded corners onthe line that is generated.

In block 814, the emphasis facility edits the markup language documentto insert the three vector graphics created in blocks 808-812. Theemphasis facility may insert the graphics in any suitable manner.

In some embodiments, for example, the emphasis facility may edit themarkup language document to indicate that the polygon vector graphiccreated in block 808 is to be displayed as a background graphic of theset of related content elements, such as by setting a “background”attribute on an existing markup element. In some embodiments,“background” attributes of a CSS markup element may be used to set thepolygon vector graphic as a background element. Such attributes maysupport the display of multiple background images or graphics together,in a vertical stack, with the first-specified being the lowest in thestack. In embodiments that use such attributes, the emphasis facilitymay set the polygon vector graphic to be displayed as the onlybackground graphic, if the emphasis facility determines that nobackground attribute was previously set, or to be displayed as thelowest background graphic in the stack, if the emphasis facilitydetermines that a background attribute already specifies one or moregraphics to be used. For example, the emphasis facility may set a“background-image” attribute to the polygon vector graphic byrepresenting the vector graphic as a Data URI, including by prependingthe polygon vector graphic to a comma-delimited list to identify thepolygon vector graphic as the lowest-level background image if one ormore background images were previously set. Other background attributesmay also be set in some embodiments that use such background attributes.For example, a “background-repeat” attribute may be set to no repeat,“background-position” may be set to a desired position at which todisplay a background image, which in this case may be set to a top-leftcorner of a bounding box for the set of related visible content elements(or to any other suitable position), a “background-attachment” attributemay be set to scroll, a “background-origin” attribute may be set to‘border-box’, a “background-clip” attribute may be set to ‘border-box’,and a “background-size” attribute may be set to ‘auto auto’. As analternative to setting some of these attributes individually, a“background” attribute may be used to set some of these attributescollectively. It should be appreciated that the values specified forthese attributes are only examples, and that other values may be used todisplay a polygon vector graphic as a background on a set of relatedvisible content elements as above. In the case that the “background”attribute is set, the emphasis facility may ensure that a“background-color” attribute is not changed from an original setting. Insome cases in which a background attribute is used, a viewingapplication may not permit the background (here, the polygon vectorgraphic) to extend by any amount beyond the bounding box of the set ofvisible content elements. It is in these embodiments that, to provide toextra padding highlighting beyond the very edge of the visible contentelements of the set, that the second vector graphic of block 810 may beused.

In other embodiments, however, the emphasis facility may insert each ofthe new vector graphics as a new element that is positioned precisely atthe position of the set of visible content elements and to be displayedbehind those elements but in front of surrounding elements, which isknown as an overlay image. The emphasis facility may configure theoverlay images with z-indexes (which regulate which visible contentelements are in front of or behind the overlay images) that set the newoverlay images on top of each of the visible content elements of theset, or behind each of the visible content elements of the set, throughevaluating the z-indexes of the elements in the set. By insertingoverlay elements that are positioned behind the set of related visiblecontent elements, the new overlay elements may be perceived asbackground on those elements with the border surrounding the elements.With transparent coloring, an overlay element positioned on top of theset of related visible content elements may also appear to be backgroundhighlighting, but may also partially obscure the content. When anoverlay image is inserted, the emphasis facility may insert the markupelements for the overlay image at a part of the markup language documentthat is different from a part of the markup language document describingthe set of visible content elements. For example, the emphasis facilitymay insert the new markup elements at an end of the markup languagedocument. This may be advantageous in some embodiments, as placing thenew markup elements at an end of the document may avoid inadvertenterrors with scripts or other elements of the markup language documentthat depend on a precise arrangement of markup elements and do notaccount for insertion of new markup elements.

In some embodiments, the emphasis facility may use criteria to selectbetween specifying the polygon vector graphic as a background image oras an overlay. In the case that the selection facility has selectedmultiple top-level elements rather than a single one (as discussedbriefly above following the discussion of FIG. 6), an overlay image maybe used to allow two display areas for the two top-level elements to besmoothly merged, as discussed below. As another example, if the set ofrelated visible content elements only includes images, such as one ormore images, an overlay may be used, or no background highlighting maybe used as the background highlighting would not show through theimage(s).

In the case that overlay images are used, the dimensions of the polygonexactly equaling the dimensions of the bounding area of the set ofrelated content elements may not be as important and, accordingly, thedimensions of the polygon may be slightly expanded and the second vectorgraphic of block 810 may not be used.

The process 800 ends following the editing of block 814.

FIG. 8B illustrates the arrangement of three vector graphics of theexample of FIG. 8A, including the polygon in the middle surrounded by afirst line element and a second line element. For ease of illustration,the first line element is shown with a different color than the polygon,though in some embodiments the first element may have the same color asthe polygon to provide an appearance of a single highlight provided bythe two vector graphics.

FIG. 8C illustrates the highlighting applied to a paragraph of text in aweb page to visually separate the text from surrounding visible contentelements. As shown in FIG. 8C, the shape of the polygon and the twolines is such that the highlighting extends only to cover the text andnot the image located to the right of the text.

As mentioned above in connection with selection techniques, in somecases a selection facility may identify multiple top-level objects thatidentify the set of related visible content element, which are thechildren of those multiple top-level objects. In this case, theidentification of the bounding box as in block 802 of FIG. 8 may beslightly adjusted. The emphasis facility may, in some cases, simplyidentify a large rectangular bounding box that encompasses all of thevisible content elements that are children of each of the top-levelelements. In other embodiments, a more complex process may be used. Thesame rectangular bounding box of block 802 may be determined for each ofthe top-level elements, and the process of blocks 804-806 for editingthe rectangular boxes to exempt surrounding visible content elements mayalso be performed for each of the rectangular bounding boxes.Subsequently, the emphasis facility may determine whether any of thebounding boxes determined in this manner overlap. Any suitable processmay be used, including a ray casting process such as the ray-castingprocess described in the web page available athttp://stackoverflow.com/questions/217578/point-in-polygon-aka-hit-test,which is incorporated herein by reference in its entirety. Ifoverlapping bounding boxes are found, then each such overlappingbounding box is merged into a polygon that is the combination of thosebounding boxes. Any suitable polygon merging process may be used. As oneexample, each of the corners of two bounding boxes to be merged may beidentified, and corners of one that are disposed within the other may beremoved. The list of corners may then be sorted according to the processdescribed in the web page available athttp://stackoverflow.com/questions/17862162/sort-anticlockwise-the-points-of-rectilinear-polygon#answer-17863840,which is incorporated herein by reference in its entirety. Through thisprocess, the corners are sorted by y-coordinate and, for matchingy-coordinate, by x-coordinate. This sorted list of coordinates willdescribe a set of edges and the merged polygon is described by theedges. The process may be repeated with pairs of polygons for anyoverlapping polygons until there are no more overlapping polygons. If atthat time there is only one polygon, the remainder of the process 800may be performed using the dimensions and shape of the merged polygons.If, however, two or more polygons remain, then the process 800 may becompleted with two or more polygons and each vector graphic created bythe emphasis facility may include two or more shapes (e.g., two or morehighlighting polygons or two or more lines), each set based on thedimensions of the non-overlapping polygons. In such a case, an overlayimage may be used, as it may be best to position the overlay image ontop of the content to be highlighted with the vector graphics havingsemitransparent coloring, to prevent any clipping that might arise fromsetting multi-polygon vector graphic as a background image or as behindother content.

In some embodiments, in addition to or as an alternative to highlightingweb content, the emphasis facility may magnify the set of relatedvisible content elements. An example of magnification is shown in FIGS.1C-1D. As shown, the set of related visible content elements (which maybe one visible content element, as in the example) are excerpted fromsurrounding visible content elements and displayed superimposed on theoriginal content, with an increased size within a border and with adifferent background graphic. The border and different backgroundgraphic may give an impression to a viewer that the magnified contenthas been magnified out of the original page, similar to an effect that atraditional magnifying glass might have on a paper being magnified or aneffect that a traditional software lens may give on a computer display.

FIG. 9 illustrates an example of a process that may be used in someembodiments to magnify a set of related visible content elements. Theprocess 900 begins in block 902, in which the emphasis facility sets asize and position of a “lens” for particular content. The size andposition of the lens may be set based on a variety of factors, includinguser input or user configuration, the size of the display area of theviewing application, the size or position of the visible contentelements to be magnified. With respect to user configuration, the usermay specify an amount of magnification to be applied, which may affect asize of the lens that is to display that magnified content. With respectto the size of the visible content elements to be magnified, theemphasis facility may be configured to, when possible, set a size of thelens such that all of the magnified content fits within the lens and thecontent does not need to be scrolled within the lens or otherwise extendoutside of the display area of the viewing application. The emphasisfacility may also be configured not to set the size of the lens to be asbig as the display area of the viewing application in either width orheight, to allow some of the surrounding content to be displayed beyondthe edges of the lens. The emphasis facility may attempt to set thewidth of the lens such that the lens is wide enough to allow visiblecontent elements of the set that were originally arranged horizontallyto be arranged horizontally within the lens, or otherwise prefer theexisting layout of visible content elements of the set. Though, theemphasis facility may also in some embodiments limit a width of thecontent, to prevent a user needing to read very wide content.

With respect to position of the lens, each of the left, right, top, andbottom sides of the lens may be calculated based on the correspondingsides of the bounding box for the set of related visible contentelements and on the desired size of the lens calculated as above. Forexample, a coordinate for a left side of the lens may be determined bystarting with the coordinate of the left side of the bounding box andthen subtracting half of the difference between the width of thebounding box and the desired width of the lens. If that calculatedcoordinate is farther to the left than a minimum left-hand coordinate,the calculated coordinate is set to the minimum left-hand coordinate. Onthe other hand, if the calculated coordinate plus the desired width ofthe lens leads to a right-hand coordinate that is farther to the rightthan a maximum right-hand coordinate, the left-hand coordinate may beshifted to allow the right side to be at the maximum right-handcoordinate. Through these calculations, the emphasis facility may locatethe lens at a position related to the original position of the content.

In block 904, the emphasis facility may change a formatting for the setof visible content elements to be magnified. Any of the examples offormatting changes described above in connection with FIG. 7 may be madeto visible content elements to be magnified in connection with theprocess 900. For example, font properties of text may be adjusted beforemagnification such that the magnified text is in a font that is easierto read. As another example, colors may be adjusted such that thevisible content elements are in colors that are easier to perceive. Anysuitable changes may be made to the content.

In block 906, the emphasis facility edits the markup language documentto display the set of related visible content elements within the lensand with an increased display size while maintaining surrounding visiblecontent elements in their original sizes. The magnification may also becarried out so as to preserve an original position and layout of thesurrounding visible content elements. Examples of ways in which themagnification may be carried out are described in detail in thefollowing paragraphs.

In addition to magnifying the set of related visible content elements inblock 906, in block 908 the emphasis facility may edit the markuplanguage document to insert an overlay image (e.g., an SVG image orother image) that is partially transparent and has a gray color or othercolor. The emphasis facility may edit the markup language document todisplay this new overlay image over all content except for the lens,through adjusting the z-index of the new overlay image or in any othermanner. By inserting this overlay image, the emphasis facility mayde-emphasize surrounding visible content elements not shown in the lens.The de-emphasis in this case may be effected by “graying out” thesurrounding content with the new overlay image. Such graying out isshown in the example of FIG. 1D, as compared to the original form shownin FIG. 1C.

Once the emphasis facility edits the markup language document in blocks906 and 908, the process 900 ends.

The emphasis facility is not limited to performing the magnification ofvisible content elements in any particular manner. Two examples of waysin which the magnification may be performed are described below, but itshould be appreciated that others are possible.

One way that the emphasis facility may magnify the set of visiblecontent elements while maintaining surrounding visible content elementsin the same size and at the same positions is to duplicate the set ofvisible content elements to be magnified. Once the selection facilityhas identified the set of visible content elements by identifying acontainer for the visible content elements, the emphasis facility maycopy from the markup language document all markup elements (e.g., HTMLelements, CSS elements, etc.) that are contained within the containerand thus relate to the set of related visible content elements. Theemphasis facility may then insert into the markup language document newmarkup elements defining a new container that has the border andbackground as illustrated in FIG. 1D. The emphasis facility may theninsert into the new container all of the duplicated markup elements. Theemphasis facility may, at that time, also adjust any spacing or positionproperties of the duplicated markup elements such that the visiblecontent elements described by the duplicated markup elements arepositioned within the lens. By duplicating the markup elements in thismanner, the emphasis facility also duplicates the definition anddescription of the set of related visible content elements, and thusinserts into the markup language document a duplicated set of therelated visible content elements. The emphasis facility may thenconfigure the new container, and all elements disposed therein, to bedisplayed with an increased size. For example, the emphasis facility mayinsert a markup element (e.g., a CSS 3.0 markup element), or modify anexisting markup element to insert a new attribute or modifying anexisting attribute, to specify that a scale transform is to be appliedto the new container to display the visible content elements within thatcontainer with an enlarged size.

The duplicating method described in the preceding paragraph isadvantageous in that it allows for magnifying content without needing toaccount for position and layout of surrounding visible content elements,and may thus be simpler. However, it may be disadvantageous in somecases as it may prevent scripts or some other content of the markuplanguage document from working properly while the lens is displayed. Forexample, when the visible content elements are duplicated, theidentifiers of tags within the duplicated region may also be duplicated.The duplicated identifiers may lead to scripts or other original contentwithin the markup language document not being able to appropriatelyinteract with content using the identifiers, as the viewing applicationmay be unable to definitively identify the content element to which tosend a message or to which to make a change, due to the presence of twocontent elements sharing an identifier.

In some embodiments, the emphasis facility may implement an alternativeapproach in which the set of visible content elements is magnified inplace. In some implementations of this alternative approach, theemphasis facility applies styles and executable code to the set ofrelated visible content elements (e.g., by applying that style and codeto each top-level element selected by the selection facility), tomagnify the visible content element and/or add visual enhancements, andto position the magnified (and otherwise edited) visible contentelements above other content as shown in the example of FIG. 1D.

Some embodiments use a list of style rules that can be applied to thelens content. These style rules can include, for example:

-   -   1. Content enlargement styles such as the scale transform,        changes to fonts, and/or the CSS zoom attribute.    -   2. Styles which implement additional visual enhancements, such        as any enhancements described above in connection with FIG. 7.    -   3. The width, height and/or max-height of the lens, which may be        adjusted according to sizing rules described in connection with        FIG. 9 and below.    -   4. Overflow and/or scrolling styles which may be used to        accommodate fitting content in the visual space provided, and/or        scrolling to overflow content.    -   5. Use of borders, outlines and/or shadows to create visual        separation with other content not in the lens view. However,        when there are multiple top level elements in the selected        content (as described above), using these styles may cause        visibly overlapping rectangles. To avoid undesired overlaps in        the case of multiple top level elements, some embodiments may        add borders, outlines and shadows via one or more overlays,        which are described below.    -   6. Changes to the background color, which may be especially        useful if the original background color uses a transparency        effect. The new background color may remove the transparency so        that underlying content does not show through the lens.    -   7. Addition of padding, for example in the same color as the        background, to create visual separation from borders, outlines        and/or shadows used.        As a result of the emphasis facility applying these styles to        the set of related visible content elements, the width and        height of the lens content may change as the browser reflows        content following the changes.

In some embodiments, the emphasis facility may position the lens toappear in a specific location. Some examples of lens positioningcriteria are described above. To position the lens, for example, theemphasis facility may set any or all of the following CSS properties onthe set of visible content elements: left, top and position. Theemphasis facility may set the position property according to one of thefollowing:

CSS position Coordinate system type (Origin point that left and topproperties are relative to) relative top-left point where the contentwould have been positioned in the layout of the document, if it had usedstatic positioning. fixed top-left of the viewport absolute If anancestor uses CSS positioning other than static positioning, thetop-left point for that ancestor Otherwise, the top-left of the documentelementAny of the above position types can be used. With any of these types,that content can be restyled without the insertion of additional DOMnodes in the middle of the content, and thus compatibility with thecontent's styles and executable code can be preserved.

The inventors have recognized that the choice of positioning type mayaffect the following:

-   -   How the left and top properties values to be set may be        calculated (as what they're relative to may change as described        in the table).    -   The amount of left and top spacing adjustment applied. This        spacing adjustment may help prevent the underlying content from        shifting as the lens width and height are altered from the        original content.        In some embodiments, the emphasis facility may compute the        position not only by calculating the difference between a        desired position and a position determined from the above table.        In some embodiments, the emphasis facility may account for the        use of any scale transform applied to the content. For example,        if the size of the content is doubled in the lens, then the        emphasis facility may determine the position of the lens by        dividing by two the amount of pixels by which the emphasis is to        be moved to the left.

In some embodiments, the emphasis facility may set the size of items inthe lens content using the CSS scale transform. For instance, to doublethe size of the content, the emphasis facility may set the scaletransform to 2. In some embodiments, the emphasis facility may makefurther adjustments, in addition to scaling the content. One reason maybe that it may be desirable in some embodiments to adjust the aspectratio of content in the lens from the original aspect ratio, such as todecrease the length of lines of text (e.g., because shorter lines may beeasier to read).

To accommodate these types of changes, the emphasis facility may makeany suitable adjustments to the size of the lens. In one example, theemphasis facility may use the following approach:

-   -   1. Use CSS width to set the desired width of the lens. The        actual value to set may be the desired width in pixels, divided        by the lens magnification level (e.g. 2 for 2× magnification).    -   2. Set the CSS height to “auto”    -   3. Set the CSS max-height to the amount of height the lens is        allowed before it breaks the sizing rules described in section        4.5, e.g. extends beyond the bottom of the screen.

It should be appreciated that the emphasis facility is not limited tousing left/width and top/height to set the size and position, and anyother suitable technique may be used. As another example, setting any 2of 3 the properties in the below chart may set the position and size fora given dimension.

Set 2 out of 3 of the following Dimension properties to set the positionand size Horizontal left, width, right Vertical top, height, bottom

In some cases, style changes on the lens may affect layout of underlyingcontent. For example, if the choice of positioning for the lens (asdiscussed above) is either absolute or fixed, this may cause the set ofrelated visible content elements, to be displayed in the lens, to beremoved from the layout flow. This may cause the surrounding content tofill in the space previously occupied by the set of visible contentelements, but it may be undesirable to allow surrounding contentelements to change position. As another example, as part of themagnification the emphasis facility may allow the set of related visiblecontent elements to change in size as described briefly above and inmore detail below. This change in size could cause the set of relatedvisible content elements to require a different amount of space, whichwould normally require the surrounding content to reflow around theexpanded set of related visible content elements.

In other embodiments in which the emphasis facility inserts a graphicaltransform, such as the CSS 3.0 scale transform, to cause themagnification of the set of related visible content elements, thetransform may not affect the amount of area that the viewing application(e.g., the web browser) allocates to the set of related visible contentelements. As such, use of this transform may not cause changes to thepositioning of the surrounding content elements that would require theemphasis facility to make counteracting adjustments to prevent changesin position. For example, magnifying the set of related visible contentelements to be displayed in the lens using the CSS 3.0 scale transformmay cause the browser to increase the size of the set of related visiblecontent elements without changing the positioning of surrounding visiblecontent elements in response. This may result in a desirable effect ofthe lens content emerging out of and covering up some of the surroundingcontent, provided that adjustments are made, as discussed below, toensure that the set of related visible content elements are configuredto display on top of the surrounding visible content elements, such asthrough adjusting z-index, and will not be displayed below thesurrounding content.

However, in other embodiments, other implementations of magnificationand/or other enhancements to the lens content, such as the addition ofpadding, borders and margins, font changes, etc., may cause the viewingapplication (e.g., the web browser) to change the amount of space thatit allocates to the set of related visible content elements. When theviewing application changes this amount of allocated space, this willcause changes to the positioning, text flow, etc., of surroundingcontent.

It may be desirable for the emphasis facility to ensure that surroundingvisible content elements of the markup language document are notaffected by the magnification, such that they maintain their originalposition, size, or other attributes. To prevent shifting the layout ofsurrounding visible content elements, the emphasis facility may beconfigured to make adjustments to the set of related visible contentelements to be magnified and/or to the surrounding visible contentelements. For example, the emphasis facility may:

-   -   1. Calculate the amount of horizontal and/or vertical shift that        would be introduced by a change in magnification or position of        the set of related visible content elements to be magnified    -   2. Add and/or remove space around the set of related visible        content elements to make up the difference.

In embodiments that use this technique, the emphasis facility maycalculate these shift values in any suitable manner. In someembodiments, to calculate a vertical spacing adjustment, the emphasisfacility may sum the amount of vertical space (e.g., margin, border andpadding) that the browser will allocate to the set of related visiblecontent element. The emphasis facility may determine the vertical spacesto be summed, for example, using processes described above fordetermining a bounding area (e.g., a bounding rectangle or boundingpolygon), which may include querying for the size of any applicablemargins, borders, padding, etc. as described above. The emphasisfacility may calculate this sum twice: once before applying stylechanges to the set of related visible content elements, and again afterthe emphasis facility has applied new styles to the set of relatedvisible content elements as part of magnifying the set and display it inthe lens. The difference between the two values may be the amount theview would shift without a space adjustment being made. Therefore, aspace adjustment may be made as the reverse of this difference. The samelogic may be applied to the horizontal dimension. Specifically, thecalculation may determine the adjustment needed to keep the underlyingcontent's horizontal positioning without shift. The only change may bethat horizontal values are utilized in this case.

In addition to this basic space adjustment logic, additional computationcan be applied in some embodiments to handle the following specialcases:

-   -   Element takes more/less actual space than set explicitly by CSS        styles: This may occur when the lens has a CSS float property.        Processes described above in connection with FIG. 8 may be used        to define the actual dimensions of the bounding area of the set        of related visible content elements.    -   The browser slightly changes dimension values such as margins,        paddings, width, etc.: This may arise in some browsers that        support a CSS zoom property. Briefly, an example of this special        case is that when a border has a width of 3 pixels with a zoom        of 1.0 (i.e., no zoom), that border might evaluate to a slightly        smaller value, such as 2.999 pixels, when zoom=1.1. One solution        may be to calculate the difference between the dimension styles        before and after the lens work. Then the difference in        height/width is may be added (or subtracted) to (from) the final        styles.    -   An element adjacent to the lens has smaller margins than the        current element: Because the effective margin between two        elements may be computed as the largest margin assigned to        either, layout may be changed if the one of these elements is        removed (e.g., when one of the elements is included in the set        of visible content elements to be magnified and is removed from        the layout). This may complicate the basic algorithm of spacing        adjustments calculations as follows: if the lens's top margin        becomes less than the above element's bottom margin, then which        element's margin takes precedence may shift to the other        element. One solution may be to detect the bounding elements and        compare the margins. If the set of visible content elements has        a greater margin than adjacent visible content elements, then        the margins on the set of related visible content elements may        be preserved without changes.

As the part of the exemplary processes described above, the boundingelements around the set of related visible content elements to bedisplayed in the lens may be detected in any suitable way.

As one example, the following process may be used in some embodiments tofind the bottom and right bounding elements for a given element:

-   -   1. Start with current element    -   2. Does the next sibling of the current element exist?        -   Yes: Set the current element to the next sibling        -   No: Set the current element to the parent. If null or            <body>, stop.    -   3. If the current element is hidden, go to step 2    -   4. Is bottom bounding element defined?        -   Yes: Skip to step 6    -   5. Is current element's top>=lens element's bottom?        -   Yes: Set bottom bounding element to current element    -   6. Is right bounding element defined?        -   Yes: Skip to step 8    -   7. Is current element's left>=lens element's right?        -   Yes: Set right bounding element to current element    -   8. If bottom or right bounding element is still undefined,        repeat step 2

A similar process may be used to find the top and left boundingelements:

-   -   1. Start with current element    -   2. Does the previous sibling of the current element exist?        -   Yes: Set the current element to the previous sibling        -   No: Set the current element to the parent. If null or            <body>, stop.    -   3. If the current element is hidden, go to step 2    -   4. Is top bounding element defined?        -   Yes: Skip to step 6    -   5. Is current element's bottom>=lens element's top?        -   Yes: Set top bounding element to current element    -   6. Is left bounding element defined?        -   Yes: Skip to step 8    -   7. Is current element's right>=lens element's left?        -   Yes: Set left bounding element to current element    -   8. If top or left bounding element is still undefined, repeat        step 2

Once the emphasis facility determines the bounding elements, thebounding elements may be used to compute the effective margin, which maybe the larger of the two adjacent margin values. For example, the marginabove the lens may be the larger of: the top margin of the lens element,and the bottom margin of the top bounding element.

Having calculated the amount of horizontal and vertical shift, theemphasis facility may use these values to stabilize the page by removingspace when the shift direction was positive or adding space when theshift direction was negative. Any suitable technique may be used toperform the space adjustment. Some examples are described below.

Space Adjustment Technique #1: Insert Placeholder Elements into the DOM

In this technique, the emphasis facility may insert an extra DOM elementbefore or after the content which the lens has been applied to. The newelement may have no visible content other than empty space, and may bethere only to add or remove space. Space may be added by creating andstyling the element to take up the desired extra space. Inserting extranodes in the middle of a markup language document, however, maysometimes break styles or executable code applied to markup languagedocument. For example, scripts incorporated into a markup languagedocument may be written to be applied to every child, the first child,nth child or last child of a parent element. This code is generallywritten assuming that no other software will change the markup languagedocument by inserting new content. Adding new children may thereforecause unintended consequences when additional children are present.

Space Adjustment Technique #2: CSS :before and/or :after Pseudo Elementsas Placeholders

In this technique, the spacing changes may be applied to apseudo-element created by browsers when a :before or :after style ruleis present. This technique may have the advantage that “pseudo” elementsare handled differently—not like any regular elements—by browsers. Theyare not natural DOM elements which can be found by selectors and, thus,may not break either page styles or scripts.

To apply this approach, in some embodiments the emphasis facility maydetermine whether to use a :before or :after pseudo element, and maydetermine the target element to apply the pseudo element to.

Does next sibling of lens Pseudo element to use on element exist? Targetelement target element Yes Next sibling :before No Parent :after

Next, the emphasis facility creates the :before or :after pseudo elementfor the target element. To do this, the emphasis facility may generate astyle rule that applies to the target element. To do this, the emphasisfacility may generate a CSS selector for this rule. One approach may beto set an attribute on the target element, and create a selector basedon that. For example, some embodiments may set an attribute calleddata-placeholder on the element, and later remove it. Next, to add:before or :after to it, the emphasis facility may generate a style rulesuch as the following and insert it into the document:

[data-placeholder]:before { content: “”; display: block; position:static; height: 30px; width: 50px; }

-   -   To remove space, the inserted style rule can apply a negative        margin to the pseudo element.

Space Adjustment Technique #3: Adjustments to CSS Margin Properties ofElements

In this technique, the emphasis facility does not generate a placeholderelement. Instead, the emphasis facility adds or removes space using CSSmargin properties. Before applying any styles to the element, theemphasis facility saves all of the original styles, including margins,so that they can be restored later. The emphasis facility may calculateshift values as explained above and may subtract those values from theoriginal margin values of the set of related visible content elements:vertical offset—from the initial top and/or bottom margin(s); horizontaloffset—from the original left and/or right margin(s).

In some embodiments, the emphasis facility may set additional CSSproperties on the set of related visible content elements and/orancestors of the set, to prevent other elements from rendering on top ofthe related visible content elements of the set or clipping any of therelated visible content elements. The emphasis facility may do this inany suitable manner, including by any or all of the followingtechniques:

-   -   z-index: setting the z-index may help to ensure that the lens is        rendered on top of other content in the document (i.e.,        appearing closest to the user).    -   overflow: if an ancestor sets the overflow it may clip the lens        to the size of that ancestor.    -   The z-index may be set when an element or its ancestors contain        styles, including z-index and others, which affect the stacking        context, or painting order, that an element belongs in.

Visually, the viewing application determines the painting order usingthe position of elements along a third axis typically referred to as the“z-axis.” Stacking contexts lie along this “z-axis” and are formattedone on top of the other. The default stacking context is effectivelyusing a z-index of 0. The z-index of each stacking context is expressedas an integer representing the order for rendering. Elements with higherstacking context appear to be closer to the observer.

Other CSS properties besides z-index may affect the stacking context. Ifthe stacking context hasn't been created explicitly yet then: opacity,CSS non-static position, and 3d transform properties will force creationof a new stacking context.

Another factor may be that ancestors control the stacking contexts oftheir descendants. As a result:

-   -   Descendants are ensured to never have a higher stacking context        than any ancestor that sets the stacking context.    -   To get the true stacking context for an element, some        embodiments may find the smallest z-index in all ancestors of        that element. The effect of other properties on the stacking        context may still be constrained by z-index values in the        ancestors.    -   Showing an element above all content may involve altering the        z-index on ancestors that have a z-index set.

In the case of the lens, if the visible content elements have a lowstacking context, the content will be partially displayed behind otherelements on the page. However, it may be desirable for the visiblecontent element of the lens to appear over all other elements. One wayto fix this may be to set the z-index on the visible content elements ofthe lens, or on the top-level element(s) that were selected by theselection facility and by which the visible content elements wereselected. The effect of ancestor z-index values as described above (item#1 from that list) may come into play in this technique.

Another technique that may be used is to change the z-index of therelated visible content elements (and/or the top-level element(s)) andof all ancestors, or at least on elements that set the z-index. CSSoverflow could also cause the lens to be partially hidden when usingthis technique, in a way that can look similar to the above screenshots.

Another exemplary implementation of the lens is to set the z-index andoverflow as follows:

‘z-index’: <high-z-index> /* On lens element and ancestors */‘overflow’: ‘visible’ /* On lens ancestors (overflow should be set toscroll on lens itself in order to allow scrolling of the lens content)*/

Applying z-index and overflow on every ancestor may break the layout ofsome content, by causing ancestors to be painted on top of elements theyshould appear underneath or by removing expected clipping. Therefore,some embodiments may utilize an algorithm that minimizes changes to theDOM. This may have the additional benefit of providing betterperformance.

To minimize alteration of z-index and overflow on ancestors, it may beuseful to address the two separately. Any suitable technique(s) may beused, some examples of which are provided below.

Minimizing z-Index Changes

To minimize z-index changes, in some embodiments the emphasis facilitymay only set the z-index on ancestor elements that actually change thestacking context and overlap the lens view. If there are no ancestorsthat force a stacking context, the emphasis facility may not set thez-index for any ancestors at all. Alternatively or additionally, theemphasis facility may set the z-index value of the lens and ancestors aslow as possible, so that ancestors do not need to cover elements theyshould be underneath.

-   -   1. For each ancestor of the lens, get the computed z-index and        store with the ancestor in an ordered JavaScript collection. Let        us call this the ancestor-zindex list.    -   2. For each item in the stored list of ancestors and z-indexes:        if the z-index of an ancestor is no different from the stored        z-index of that ancestor's parent, then remove that element from        the ancestor-zindex list.

This provides an ancestor-zindex list structured like the following:

<div>, z-index=30 <table>, z-index=10 (descendant of div) *lenselement*, z-index=MAX_ZINDEX (descendant of table)

The emphasis facility may be configured with MAX_ZINDEX set to themaximum possible value for the z-index in the web browser being used.This may be the largest signed 32-bit integer, 2147483647.

Next, the emphasis facility may obtain a list of all elements that setthe stacking context in the underlying content. This can be achieved inany suitable way, such as via brute force (for each element, compare thez-index against its parent element's z-index, to see if has changed), orby parsing the content's style sheets. In this case, any element thataffects the z-index may be checked. The emphasis facility may computethe effective z-index by including anything that can affect the stackingcontext, such as z-index, opacity and transform properties.

Next, for each element that sets a z-index, and is not an ancestor ofthe lens, the emphasis facility may carry out the following process:

-   -   1. Check if the element's bounding rectangle intersects with the        lens. Specifically, the rectangles intersect if        Lens.left<Element.right AND Lens.right>Element.left AND        Lens.top<Element.bottom AND Lens.bottom>Element.top.        -   If they intersect, go to step 2, else continue the loop with            the next item (repeat step 1).    -   2. Find the first item in the ancestor-zindex list that has a        lower z-index than the intersecting element's z-index. If none        do, continue the loop with the next item (back to step 1).    -   3. Show the intersecting element in the ancestor-zindex list in        front of the other intersecting element as follows:        -   Set the z-index of the ancestor-zindex list item's element            so that it's equal to the z-index of that intersecting            element. Alternatively, reverse this, and set the z-index of            the intersecting element so that it's the same as the item            in the ancestor-zindex list.    -   4. Go back step 2, starting at the next position in the        ancestor-zindex list.    -   5. Continue loop with next item (back to step 1)

Minimizing CSS Overflow Changes

To minimize CSS overflow changes, the emphasis facility may not changeoverflow properties when the bounds already encompass the set of relatedvisible content element, because no clipping would occur. For example,the emphasis facility may carry out the following process:

-   -   1. Set current element to the top-level element selected by the        selection facility as representing the set of related visible        content elements.    -   2. Set current element to parent of current element.    -   3. If the horizontal bounds of the current element do not fully        encompass the horizontal bounds of the lens, set overflow-x:        visible.    -   4. If the vertical bounds of the current element do not fully        encompass the vertical bounds of the lens, set overflow-y:        visible.    -   5. If either overflow-x or overflow-y has been altered, go back        to step 2, else stop.

In some embodiments, the emphasis facility may apply animation to theset of related visible content elements to be magnified in the lens,allowing size, positional and other style changes to be appliedgradually over a time period, for example over a period of 400 ms. Thismay make the visual changes easier for the user to understand, as theymay occur smoothly and not all at once.

In embodiments that use such animation, any suitable animationtechnique(s) may be used. As one example, the emphasis facility may usethe following process to implement a transition animation:

-   -   0. Preparation step: Store the initial styles of the element.        -   Store the current style values for the element. These styles            can be restored at a later time in order to return the view            to its original state where the lens was hidden/off.    -   1. Apply positioning & stacking context.        -   Using the transform property is explained in step 2.2. The            current step is preparation for it. If the transform            property has a value different than none, a stacking context            will be created implicitly. In that case the object will act            as a containing block for position: fixed elements that it            contains. To have full control over animation the position            may be set explicitly and the new stacking context created.    -   2. Prepare and set any pre-lens styles that can be pre-set        before animation starts.        -   For example, border width animation may not be necessary.            Some embodiments may set the new value for border width            before the animation starts. Some other properties that may            be set for the lens include, for example:            -   constrained width of lens;            -   expanded height of lens;            -   negative or positive margins to compensate the                underlying content shift which was possibly caused by                one of the two or both of previous styles;            -   contrast background color;            -   etc.    -   3. Build the full animation transition stack.        -   Due to the actions described in step 2, in some embodiments            the current step may set as few properties as possible.        -   Some exemplary methods for applying animations include            using:            -   Raw JavaScript to iteratively apply the property changes            -   Animation features in popular JavaScript frameworks such                as jQuery            -   CSS Transitions provided in modern browsers, which allow                property changes in CSS values to occur smoothly over a                specified duration. In this case, supporting older                browsers may be achieved by adding the vendor-supplied                prefixes such as -moz and -o prefixes as well as the                actual CSS3 transition property. The non-prefixed                properties may be added last in the stack to ensure that                the final implementation will trump the others as the                property becomes standardized in newer browser versions.    -   4. Trigger the animations        -   This may be done via a method call to begin the animations            or by setting an attribute that activates the CSS transition            properties.

As discussed above in connection with FIG. 9, in some embodiments theemphasis facility may alter the display of visible content elementsoutside of the lens. For example, the emphasis facility may alter theview of the surrounding visible content elements to de-emphasize thesurrounding visible content elements and provide a clear visualindication of the difference between lens and underlying content, and/orto bring the set of related visible content elements of the lens to theforeground with as little distraction as possible. Examples ofalterations the emphasis facility may make include:

-   -   Dimming content outside of the lens. This may be generally        designed to increase the contrast between the primary and        secondary content, making the primary content visually more        apparent.    -   Blurring content outside of the lens. The same idea as with the        dimmer may be used in this item: to help draw attention to the        lens content.    -   Stopping animations or movement in underlying content, which the        inventors have recognized may be visually or cognitively        distracting.    -   Adding borders, outlines or shadows to the lens. Some        embodiments may use this approach rather adding        borders/outlines/shadows via styles on the lens as described        above, or when there are multiple top-level elements in the        lens, as described above. The borders, outlines and shadows may        thus be created as single polygons, thus avoiding overlaps. The        source polygons can be generated via standard polygon merge        algorithms (such as those described above in connection with        FIG. 8) from the bounding rectangles of top-level elements        selected by the selection facility.

In some embodiments, while the emphasis facility may make alterations tothe view of surrounding visible content elements, these alterations maybe made only to bring attention to the lens and to deemphasize theunderlying content, without changing the placement of elements in theunderlying content, as described above.

In the case of a web page, one exemplary implementation of dimmer thatmay be used by the emphasis facility is to draw an SVG element over theentire web page, with a hole drawn/clipped around the area of the lenscontent, letting the lens show through. Mouse-pointer events may beignored on the SVG container so that the user can click on the documentbelow the SVG overlay. The steps of an exemplary process that may beimplemented by the emphasis facility are listed below:

-   -   1. Create an element using <svg>    -   2. Use SVG to create a shape that covers everything on the        screen (or more, or less), with a hole cut out for the lens        content to show through    -   3. Set the color and other presentation attributes of the SVG        element    -   4. Set the opacity level of the SVG element so that the        underlying content can show through it as a child of the <html>        element    -   5. Set the z-index to the maximum value, so that it shows above        all other content    -   6. Insert the new element as a child of the <html>element in the        DOM

Using an SVG element in this way may have some potential advantages:

-   -   It can very precisely control the shape of the highlighted area    -   SVG paths look crisp/sharp when zoomed in    -   It performs faster than an HTML <canvas> approach    -   It uses minimal code to ensure that the overlay is drawn over        all underlying content, by putting it as a child of the document        itself (making the overlay a high-level element in the DOM),        which ensures it can control its own stacking context.

In some other embodiments, the emphasis facility may implement theoverlay as part of the lens itself (e.g., in the same subtree of the DOMas the lens content). In this case, additional z-index changes toancestors may be desirable in order to ensure that the overlay isrendered over all other content. These additional ancestor z-indexchanges could have the effect in some situations of causing ancestors ofthe related visible content elements of the lens to render over elementsthey were intended to be underneath.

Other relevant techniques may include inserting the SVG element in adifferent location in the DOM or as a pseudo element, or using CSSoutline, border or box-shadow to render, or any other suitabletechniques.

As should be appreciated from the foregoing, in some embodiments theselection facility and/or the emphasis facility may be implemented asscripting language code that is incorporated into a markup languagedocument, such as code that is incorporated into an HTML document. Insome such cases, the selection facility and/or the emphasis facility maybe incorporated into the markup language manually, by a developer (e.g.,a web developer) that creates the markup language document. Theselection facility and/or the emphasis facility may be incorporated by adeveloper in cases where the developer expects that a substantial numberof users of the markup language document may benefit from use of theselection facility and/or the emphasis facility, such as in a case thatmany of the users will have difficulties or disabilities in eyesight ormay be unfamiliar with the operations of computers.

In other embodiments, however, the selection facility and/or theemphasis facility may be incorporated into the markup language documentautomatically, such as by an insertion facility that is configured toinsert the selection facility and/or the emphasis facility into markuplanguage documents. The insertion facility may be implemented in anysuitable manner, as embodiments are not limited in this respect. In someembodiments, for example, the insertion facility may be implemented aspart of a proxy server or other server that relays markup languagedocuments. The proxy server may, for example, be a server that isintended for use by users who have difficulties or disabilities witheyesight, or are unfamiliar with the operations of computers. Theseusers may access markup language documents via the proxy server suchthat the proxy server relays the documents to the devices after theproxy server has edited the documents to insert the selection facilityand/or the emphasis facility, such that the selection facility and/orthe emphasis facility is available for use by the users. Such a proxyserver may be used in any suitable manner, including known techniques.As one example, a viewing application (e.g., web browser) may beconfigured to access markup language documents via the proxy server.

As another example of a manner in which an insertion facility mayoperate to incorporate the selection facility and/or emphasis facilityinto the markup language document, the insertion facility may beimplemented as a portion of a viewing application, such as a plug-in toa viewing application, that is executed on a device operated by a user.When the user operates the viewing application to download and processmarkup language documents, the insertion facility may edit the markuplanguage documents to incorporate the selection facility and/or theemphasis facility.

FIG. 10 illustrates an example of a process that an insertion facilitymay carry out in some embodiments to incorporate a selection facilityand/or emphasis facility into a markup language document. The process1000 of FIG. 10 begins in block 1002, in which the insertion facilityreceives input information regarding a markup language document. Theinformation received in block 1002 may be the markup language documentitself or information describing the markup language document, such asan address from which the markup language document may be downloaded. Ifthe insertion facility does not receive the markup language document inblock 1002, the insertion facility downloads the markup languagedocument. In block 1004, the insertion facility edits the markuplanguage document to incorporate the selection facility and/or theemphasis facility. As discussed above, the selection facility and/or theemphasis facility may be incorporated in any suitable manner, includingby being inserted entirely into the markup language document or by beingdisposed in one or more other files that is/are referenced by one ormore markup elements of the markup language document. Once the edited ofblock 1004 is completed, in block 1006 the insertion facility outputsthe edited markup language document, such as by passing the editedmarkup language document to another component of a viewing applicationto be processed or by communicating the edited markup language documentover a network. After the output of block 1006, the process 1000 ends.

FIG. 11 illustrates an example of a computer system in which someembodiments may operate. The computer system of FIG. 11 includes acomputing device 1102 that includes a data store 1102A. While computingdevice 1102 is illustrated as a single server, it should be appreciatedthat it may be implemented as any suitable one or more computing devicesthat may communicate over a communication network, as embodiments arenot limited in this respect. Data store 1102A may also be implemented inany suitable manner. Data store 1102A may store one or more markuplanguage documents. The computer system also includes a computing device1104 that may execute a viewing application that is operated by a userto download, process, and display a markup language document. While thecomputing device 1104 is illustrated in FIG. 11 as a desktop personalcomputer, it should be appreciated that embodiments are not so limitedand that the device 1104 may be implemented as a laptop personalcomputer, a smart phone, a tablet computer, a gaming device, aweb-enabled television, or any number of other devices that may beoperated by users to download, process, and view markup languagedocuments. In some embodiments, the computing device 1104 maycommunicate directly with the computing device 1102 to request a markuplanguage document stored in the data store 1102A, and the markuplanguage document returned from the device 1102 may have a selectionfacility and/or an emphasis facility incorporated therein. In otherembodiments, as discussed above in connection with FIG. 10, thecomputing device 1104 may request the markup language document via anintermediary computing device 1106. The intermediary computing device1106, which may be implemented as any suitable computing device, mayhave a data store 1106A that stores a selection facility and/or anemphasis facility. The intermediary computing device 1106 may act as arelay for markup language documents between the device 1102 and thedevice 1104 and may, as part of the relay, edit the markup languagedocument to insert a selection facility and/or an emphasis facility.Communications between the devices 1102, 1104, and 1106 may be exchangedvia a communication network 1108, which may be any suitable one or morewired and/or wireless computer networks, including local area networks,wide area networks, and/or the Internet.

Techniques operating according to the principles described herein may beimplemented in any suitable manner. Included in the discussion above area series of flow charts showing the steps and acts of various processesthat select a group of related visible content elements of at least onemarkup language document and edit the at least one markup languagedocument to emphasize those visible content elements while maintainingsurrounding visible content elements in an original format. Theprocessing and decision blocks of the flow charts above represent stepsand acts that may be included in algorithms that carry out these variousprocesses. Algorithms derived from these processes may be implemented assoftware integrated with and directing the operation of one or moresingle- or multi-purpose processors, may be implemented asfunctionally-equivalent circuits such as a Digital Signal Processing(DSP) circuit or an Application-Specific Integrated Circuit (ASIC), ormay be implemented in any other suitable manner. It should beappreciated that the flow charts included herein do not depict thesyntax or operation of any particular circuit or of any particularprogramming language or type of programming language. Rather, the flowcharts illustrate the functional information one skilled in the art mayuse to fabricate circuits or to implement computer software algorithmsto perform the processing of a particular apparatus carrying out thetypes of techniques described herein. It should also be appreciatedthat, unless otherwise indicated herein, the particular sequence ofsteps and/or acts described in each flow chart is merely illustrative ofthe algorithms that may be implemented and can be varied inimplementations and embodiments of the principles described herein.

Accordingly, in some embodiments, the techniques described herein may beembodied in computer-executable instructions implemented as software,including as application software, system software, firmware,middleware, embedded code, or any other suitable type of computer code.Such computer-executable instructions may be written using any of anumber of suitable programming languages and/or programming or scriptingtools, and also may be compiled as executable machine language code orintermediate code that is executed on a framework or virtual machine.

When techniques described herein are embodied as computer-executableinstructions, these computer-executable instructions may be implementedin any suitable manner, including as a number of functional facilities,each providing one or more operations to complete execution ofalgorithms operating according to these techniques. A “functionalfacility,” however instantiated, is a structural component of a computersystem that, when integrated with and executed by one or more computers,causes the one or more computers to perform a specific operational role.A functional facility may be a portion of or an entire software element.For example, a functional facility may be implemented as a function of aprocess, or as a discrete process, or as any other suitable unit ofprocessing. If techniques described herein are implemented as multiplefunctional facilities, each functional facility may be implemented inits own way; all need not be implemented the same way. Additionally,these functional facilities may be executed in parallel and/or serially,as appropriate, and may pass information between one another using ashared memory on the computer(s) on which they are executing, using amessage passing protocol, or in any other suitable way.

Generally, functional facilities include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types. Typically, the functionalityof the functional facilities may be combined or distributed as desiredin the systems in which they operate. In some implementations, one ormore functional facilities carrying out techniques herein may togetherform a complete software package. These functional facilities may, inalternative embodiments, be adapted to interact with other, unrelatedfunctional facilities and/or processes, to implement a software programapplication.

Some exemplary functional facilities have been described herein forcarrying out one or more tasks. It should be appreciated, though, thatthe functional facilities and division of tasks described is merelyillustrative of the type of functional facilities that may implement theexemplary techniques described herein, and that embodiments are notlimited to being implemented in any specific number, division, or typeof functional facilities. In some implementations, all functionality maybe implemented in a single functional facility. It should also beappreciated that, in some implementations, some of the functionalfacilities described herein may be implemented together with orseparately from others (i.e., as a single unit or separate units), orsome of these functional facilities may not be implemented.

Computer-executable instructions implementing the techniques describedherein (when implemented as one or more functional facilities or in anyother manner) may, in some embodiments, be encoded on one or morecomputer-readable media to provide functionality to the media.Computer-readable media include magnetic media such as a hard diskdrive, optical media such as a Compact Disk (CD) or a Digital VersatileDisk (DVD), a persistent or non-persistent solid-state memory (e.g.,Flash memory, Magnetic RAM, etc.), or any other suitable storage media.Such a computer-readable medium may be implemented in any suitablemanner, including as computer-readable storage media 1206 or 1306 ofFIGS. 12-13 described below (i.e., as a portion of a computing devices1200 or 1300) or as a stand-alone, separate storage medium. As usedherein, “computer-readable media” (also called “computer-readablestorage media”) refers to tangible storage media. Tangible storage mediaare non-transitory and have at least one physical, structural component.In a “computer-readable medium,” as used herein, at least one physical,structural component has at least one physical property that may bealtered in some way during a process of creating the medium withembedded information, a process of recording information thereon, or anyother process of encoding the medium with information. For example, amagnetization state of a portion of a physical structure of acomputer-readable medium may be altered during a recording process.

In some, but not all, implementations in which the techniques may beembodied as computer-executable instructions, these instructions may beexecuted on one or more suitable computing device(s) operating in anysuitable computer system, including the exemplary computer system ofFIG. 11, or one or more computing devices (or one or more processors ofone or more computing devices) may be programmed to execute thecomputer-executable instructions. A computing device or processor may beprogrammed to execute instructions when the instructions are stored in amanner accessible to the computing device or processor, such as in adata store (e.g., an on-chip cache or instruction register, acomputer-readable storage medium accessible via a bus, etc.). Functionalfacilities comprising these computer-executable instructions may beintegrated with and direct the operation of a single multi-purposeprogrammable digital computing device, a coordinated system of two ormore multi-purpose computing device sharing processing power and jointlycarrying out the techniques described herein, a single computing deviceor coordinated system of computing device (co-located or geographicallydistributed) dedicated to executing the techniques described herein, oneor more Field-Programmable Gate Arrays (FPGAs) for carrying out thetechniques described herein, or any other suitable system.

FIG. 12 illustrates one exemplary implementation of a computing devicein the form of a computing device 1200 that may be used in a systemimplementing techniques described herein, although others are possible.It should be appreciated that FIG. 12 is intended neither to be adepiction of necessary components for a computing device to operate as acomputing device operated by a client to view markup language documentsin accordance with the principles described herein, nor a comprehensivedepiction.

Computing device 1200 may comprise at least one processor 1202, anetwork adapter 1204, and computer-readable storage media 1206.Computing device 1200 may be, for example, a desktop or laptop personalcomputer, a personal digital assistant (PDA), a smart mobile phone, atablet computer, a server, a gaming device, or any other suitablecomputing device. Network adapter 1204 may be any suitable hardwareand/or software to enable the computing device 1200 to communicate wiredand/or wirelessly with any other suitable computing device over anysuitable computing network. The computing network may include wirelessaccess points, switches, routers, gateways, and/or other networkingequipment as well as any suitable wired and/or wireless communicationmedium or media for exchanging data between two or more computers,including the Internet. Computer-readable media 1206 may be adapted tostore data to be processed and/or instructions to be executed byprocessor 1202. Processor 1202 enables processing of data and executionof instructions. The data and instructions may be stored on thecomputer-readable storage media 1206.

The data and instructions stored on computer-readable storage media 1206may comprise computer-executable instructions implementing techniqueswhich operate according to the principles described herein. In theexample of FIG. 12, computer-readable storage media 1206 storescomputer-executable instructions implementing various facilities andstoring various information as described above. Computer-readablestorage media 1206 may store a viewing application 1208 to process andview markup language documents (e.g., a web browser to view web pages)using known techniques. Media 1206 may also store one or more markuplanguage documents 1210 that have incorporated therein a selectionfacility 1212 and an emphasis facility 1214.

FIG. 13 illustrates one exemplary implementation of a computing devicein the form of a computing device 1300 that may be used in a systemimplementing techniques described herein, although others are possible.It should be appreciated that FIG. 13 is intended neither to be adepiction of necessary components for a computing device to operate asan intermediary computing device in accordance with the principlesdescribed above in connection with FIG. 11, nor a comprehensivedepiction.

Computing device 1300 may comprise at least one processor 1302, anetwork adapter 1304, and computer-readable storage media 1306.Computing device 1300 may be, for example, a desktop or laptop personalcomputer, a server, a rack-mounted computer or other networking element,or any other suitable computing device. Network adapter 1304 may be anysuitable hardware and/or software to enable the computing device 1300 tocommunicate wired and/or wirelessly with any other suitable computingdevice over any suitable computing network. The computing network mayinclude wireless access points, switches, routers, gateways, and/orother networking equipment as well as any suitable wired and/or wirelesscommunication medium or media for exchanging data between two or morecomputers, including the Internet. Computer-readable media 1306 may beadapted to store data to be processed and/or instructions to be executedby processor 1302. Processor 1302 enables processing of data andexecution of instructions. The data and instructions may be stored onthe computer-readable storage media 1306.

The data and instructions stored on computer-readable storage media 1306may comprise computer-executable instructions implementing techniqueswhich operate according to the principles described herein. In theexample of FIG. 13, computer-readable storage media 1306 storescomputer-executable instructions implementing various facilities andstoring various information as described above. Computer-readablestorage media 1306 may store a selection facility 1308, an emphasisfacility 1310, and an insertion facility 1312.

While not illustrated in FIGS. 12-13, a computing device mayadditionally have one or more components and peripherals, includinginput and output devices. These devices can be used, among other things,to present a user interface. Examples of output devices that can be usedto provide a user interface include printers or display screens forvisual presentation of output and speakers or other sound generatingdevices for audible presentation of output. Examples of input devicesthat can be used for a user interface include keyboards, and pointingdevices, such as mice, touch pads, and digitizing tablets. As anotherexample, a computing device may receive input information through speechrecognition or in other audible format.

Embodiments have been described where the techniques are implemented incircuitry and/or computer-executable instructions. It should beappreciated that some embodiments may be in the form of a method, ofwhich at least one example has been provided. The acts performed as partof the method may be ordered in any suitable way. Accordingly,embodiments may be constructed in which acts are performed in an orderdifferent than illustrated, which may include performing some actssimultaneously, even though shown as sequential acts in illustrativeembodiments.

Various aspects of the embodiments described above may be used alone, incombination, or in a variety of arrangements not specifically discussedin the embodiments described in the foregoing and is therefore notlimited in its application to the details and arrangement of componentsset forth in the foregoing description or illustrated in the drawings.For example, aspects described in one embodiment may be combined in anymanner with aspects described in other embodiments.

Use of ordinal terms such as “first,” “second,” “third,” etc., in theclaims to modify a claim element does not by itself connote anypriority, precedence, or order of one claim element over another or thetemporal order in which acts of a method are performed, but are usedmerely as labels to distinguish one claim element having a certain namefrom another element having a same name (but for use of the ordinalterm) to distinguish the claim elements.

Also, the phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” “having,” “containing,” “involving,” andvariations thereof herein, is meant to encompass the items listedthereafter and equivalents thereof as well as additional items.

The word “exemplary” is used herein to mean serving as an example,instance, or illustration. Any embodiment, implementation, process,feature, etc. described herein as exemplary should therefore beunderstood to be an illustrative example and should not be understood tobe a preferred or advantageous example unless otherwise indicated.

Having thus described several aspects of at least one embodiment, it isto be appreciated that various alterations, modifications, andimprovements will readily occur to those skilled in the art. Suchalterations, modifications, and improvements are intended to be part ofthis disclosure, and are intended to be within the spirit and scope ofthe principles described herein. Accordingly, the foregoing descriptionand drawings are by way of example only.

What is claimed is:
 1. A method comprising: with a viewing applicationprocessing at least one markup language document, executing instructionsincorporated with the at least one markup language document to carry outacts of: determining a group of one or more visible content elements, ofat least one markup language document, to be highlighted in a display ofthe at least one markup language document; and editing markup elementsrelating to the one or more visible content elements to visuallyseparate the one or more visible content elements from surroundingvisible content elements, wherein the editing comprises: editing atleast one first markup element to specify a graphic to display as abackground of the one or more visible content elements; and editing atleast one second markup element to specify a first border surroundingthe one or more visible content elements.
 2. The method of claim 1,further comprising: determining dimensions of display areas of eachvisible content element of the group of one or more visible contentelements; and based on the dimensions of the display areas of eachvisible content element of the group, determining dimensions of a groupdisplay area that includes the one or more visible content elements ofthe group and does not include other visible content elements.
 3. Themethod of claim 2, wherein determining dimensions of the group displayarea comprises: identifying a default background of the group of one ormore visible content elements; determining a rectangular display areathat includes the display areas of each visible content element of thegroup; determining whether any other visible content element, of the atleast one markup language document, that is not a part of the group isdisposed at least partially within the rectangular display area; inresponse to determining that no other visible content element isdisposed at least partially within the rectangular display area, settingdimensions of the rectangular display area as the dimensions of thegroup display area; and in response to determining that at least oneother visible content element is disposed at least partially within therectangular display area, determining, from the rectangular displayarea, a revised display area that includes the one or more visiblecontent elements of the group and does not include the at least oneother visible content element, and setting dimensions of the reviseddisplay area as the dimensions of the group display area.
 4. The methodof claim 3, wherein: editing the at least one first markup element tospecify the graphic to display as the background comprises configuring afirst vector graphic based on the dimensions of the group display areaand editing the at least one first markup element to specify that thefirst vector graphic is to be displayed as the background of the groupof one or more visible content elements; and editing the at least onesecond markup element to specify the first border comprises configuringa second vector graphic based on the dimensions of the group displayarea and editing the at least one second markup element to specify thatthe second vector graphic is to be displayed as the border surroundingthe group of one or more visible content elements.
 5. The method ofclaim 4, wherein: configuring the first vector graphic based on thedimensions of the group display area comprises: expanding the dimensionsof the group display area by a first amount to produce a first set ofrevised dimensions covering the group display area and a first marginarea around the group display area, and specifying a size and shape of apolygon of the first vector graphic, based on the first set of reviseddimensions; editing the at least one first markup element to specifythat the first vector graphic is to be displayed as the background ofthe one or more visible content elements comprises inserting at leastone first new markup element into the at least one markup languagedocument specifying that the first vector graphic is to be displayed asthe background; configuring the second vector graphic based on thedimensions of the group display area comprises: specifying a shape of afirst line based on the first set of revised dimensions and specifying awidth of the first line; editing the at least one first markup elementto specify that the second vector graphic is to be displayed around thefirst vector graphic the one or more visible content elements comprisesinserting at least one second new markup element into the at least onemarkup language document specifying that the second vector graphic is tobe displayed alongside the first vector graphic.
 6. The method of claim5, further comprising: expanding the dimensions of the group displayarea by a second amount to produce a second set of revised dimensionscovering the group display area and a second margin area around thegroup display area, the second margin area having a width matching thewidth of the line; and wherein editing the markup elements relating tothe one or more visible content elements to visually separate the one ormore visible content elements from surrounding visible content elementsfurther comprises: specifying a shape of a second line based on thesecond set of revised dimensions and specifying a width of the secondline; and inserting at least one third new markup element into the atleast one markup language document specifying that the second line is tobe displayed alongside the first vector graphic.
 7. The method of claim6, wherein: specifying the size and the shape of the polygon of thefirst vector graphic comprises specifying the size and shape in a markuplanguage; specifying the shape and weight of the first line of thesecond vector graphic comprises the shape and the weight in the markuplanguage; and specifying the shape and weight of the second line of thethird vector graphic comprises the shape and the weight in the markuplanguage.
 8. The method of claim 7, wherein: specifying the shape of thefirst line comprises adjusting the first set of revised dimensions toinclude rounded corners for the first line; and specifying the shape ofthe second line comprises adjusting the second set of revised dimensionsto include rounded corners for the second line.
 9. The method of claim1, wherein: editing the at least one first markup element to specify agraphic to display as the background of the one or more visible contentelements comprises modifying at least one existing markup elementregarding the one or more visible content elements to set a backgroundattribute of the at least one existing markup element; and editing theat least one second markup element to specify the first bordersurrounding the one or more visible content elements comprises insertingat least one new markup element specifying the border and indicatingthat the border is to be displayed at a position surrounding the one ormore visible content elements.
 10. The method of claim 9, whereinmodifying at least one existing markup element regarding the one or morevisible content elements to set the background attribute of the at leastone existing markup element comprises, in response to determining thatthe at least one existing markup element specifies at least one firstbackground graphic of the one or more visible content elements,modifying the at least one existing markup element to specify a secondbackground graphic to display below the at least one first backgroundgraphic.
 11. The method of claim 1, wherein: editing the at least onefirst markup element to specify a graphic to display as the backgroundof the one or more visible content elements comprises inserting at leastone first new markup element specifying the graphic and indicating thatthe graphic is to be displayed at a position overlapping with and behindthe one or more visible content elements; and editing the at least onesecond markup element to specify the first border surrounding the one ormore visible content elements comprises inserting at least one secondnew markup element specifying the border and indicating that the borderis to be displayed at a position surrounding the one or more visiblecontent elements.
 12. The method of claim 1, further comprising: editingat least one third markup element to specify a second border surroundingthe first border and the one or more visible content elements.
 13. Themethod of claim 12, wherein: the at least one second markup elementspecifies a different color for the first border than the at least onethird markup element specifies for the second border; and the at leastone second markup elements specifies the same color for the backgroundand for the first border.
 14. The method of claim 1, wherein: the methodfurther comprises receiving user input indicating a first visiblecontent element of the at least one markup language document; thedetermining and the editing are performed in response to receiving theuser input; the group of one or more visible content elements comprisesthe first visible content element; determining the group of one or morevisible content elements comprises determining whether a layout of theat least one markup language document indicates the first visiblecontent element is one of a distinct set of two or more visible contentelements distinct from surrounding visible content elements; and editingthe markup elements relating to the one or more visible content elementsto visually separate the one or more visible content elements fromsurrounding visible content elements comprises, in response todetermining that the layout indicates that the first visible contentelement is a member of a distinct set, editing the markup elements tovisually separate the distinct set of two or more visible contentelements from the surrounding visible content elements.
 15. The methodof claim 14, wherein: receiving the user input comprises receivinginformation regarding movement of a cursor of a pointing device; themethod further comprises, in response to receiving the informationregarding the movement of the cursor, determining whether the cursor hasmoved outside of a display area of a previously-identified group of oneor more visible content elements for which the at least one markuplanguage document was edited to visually separate from other visiblecontent elements surrounding the previously-identified group; and thedetermining and the editing are performed in response to receiving theuser input and determining that the cursor has moved outside the displayarea of the previously-identified group.
 16. The method of claim 15,wherein: the method further comprises, in response to determining thatthe cursor has moved outside the display area of thepreviously-identified group, editing second markup elements of the atleast one markup language document relating to the previously-identifiedgroup undo prior edits to visually separate the previously-identifiedgroup from other visible content elements surrounding thepreviously-identified group; the method further comprises, in responseto determining the group of one or more visible content elements andprior to editing the markup elements relating to the one or more visiblecontent elements, storing an unedited form of the markup elementsrelating to the one or more visible content elements; the method furthercomprises: receiving a subsequent user input regarding subsequentmovement of the cursor of the pointing device; and in response to thesubsequent user input, determining whether the cursor has moved outsideof a display area of the group of one or more visible content elements;and in response to determining that the cursor has moved outside thedisplay area of the group of one or more visible content elements,undoing the editing of the markup elements to visually separate the oneor more visible content elements at least in part by editing the markupelements to restore the markup elements to the unedited form.
 17. Themethod of claim 1, wherein: the at least one markup language document isat least one document of a web page; and the viewing application is aweb browser.
 18. The method of claim 17, further comprising: with theviewing application, editing the at least one markup language documentto insert the executable instructions into the at least one markuplanguage document.
 19. The method of claim 1, wherein the instructionsincorporated with the at least one markup language document define thedetermining and the editing to be independent of the at least one markuplanguage document and applicable to other markup language documents. 20.At least one computer-readable storage medium having encoded thereon atleast one markup language document having incorporated thereinexecutable instructions, wherein the executable instructions, whenexecuted by at least one processor, cause the at least one processor tocarry out a method comprising: editing markup elements, of at least onemarkup language document, relating to a set of one or more visiblecontent elements to visually separate the set of one or more visiblecontent elements from surrounding visible content elements, the editingcomprising: determining dimensions of a display area including the oneor more visible content elements; calculating, based on the dimensions,first expanded dimensions including the display area and a first marginarea surrounding the display area; calculating, based on the dimensions,second expanded dimensions including the display area, the first marginarea, and a second margin area surrounding the display area; editing theat least one markup language document to insert a first new visiblecontent element as a background graphic of the one or more visiblecontent element, the background graphic being a vector graphicspecifying a polygon having dimensions of the display area and having afirst color; editing the at least one markup language document to insertone or more first new markup elements defining a second new visiblecontent element positioned overlapping the one or more visible contentelements, the second new visible content element being a vector graphicincluding a line having a size and shape specified according to thefirst expanded dimensions to surround the display area and having thefirst color; and editing the at least one markup language document toinsert one or more second new markup elements defining a third newvisible content element positioned overlapping the one or more visiblecontent elements, the third new visible content element being a vectorgraphic including a line having a size and shape specified according tothe second expanded dimensions to surround the display area and thefirst margin area and having a second color.
 21. The at least onecomputer-readable storage medium of claim 20, wherein: the editing isperformed in response to detecting a cursor of a pointing devicepositioned over one of the one or more visible content elements; themethod further comprises storing an unedited form of the markup elementsrelating to the set of one or more visible content elements; and themethod further comprises, in response to detecting that the cursor ofthe pointing device is no longer positioned over the one or more visiblecontent elements, editing the markup elements to restore the markupelements to the unedited form, wherein editing the markup elements torestore the markup elements to the unedited form comprises removing fromthe at least one markup language document the first new visible contentelement, the second new visible content element, and the third newvisible content element.
 22. An apparatus comprising: at least oneprocessor; and at least one computer-readable storage medium havingencoded thereon executable instructions that, when executed by the atleast one processor, cause the at least one processor to carry out amethod comprising: with a viewing application processing at least onemarkup language document, executing instructions incorporated with theat least one markup language document to carry out acts of: determininga group of one or more visible content elements, of at least one markuplanguage document, to be highlighted in a display of the at least onemarkup language document; and editing markup elements relating to theone or more visible content elements to visually separate the one ormore visible content elements from surrounding visible content elements,wherein the editing comprises: editing at least one first markup elementto specify a graphic to display as a background of the one or morevisible content elements; and editing at least one second markup elementto specify a first border surrounding the one or more visible contentelements.